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OVERVIEW 


P., stsecondary education is widely seen as a necessity in the modern economy. Yet students at com- 
munity colleges often face steep odds when it comes to completing a degree. Community colleges 
serve many low-income and first-generation students as well as students of color, all of whom must 
contend with many obstacles to success. These include the inability to pay for expenses not covered 
by financial aid, a lack of academic preparedness, a confusing array of requirements and paperwork for 
financial aid and course selection, and competing priorities such as the need to work. At the same time, 
two-year colleges are severely underfunded, and therefore are unable to provide the level of personal 
support that many students require. The three-year graduation rate for first-time, full-time freshmen in 
community colleges is only 35 percent. 


MDRC and the Detroit Regional Chamber partnered in 2016 to create the Detroit Promise Path (DPP), an 
evidence-based student services program designed to help more Detroit high school graduates—among 
the nation’s most underserved students—enroll and persist in college, accumulate credits, graduate, 
and potentially transfer to a four-year program. DPP builds on an existing scholarship program called 
the Detroit Promise, launched in 2013, which covers community college tuition and fees for up to three 
years of attendance. At the heart of DPP are campus coaches who help students acclimate to college, 
proactively reach out to them with help and reminders about tasks and deadlines, and offer a sympa- 
thetic ear to young people who may be grappling with personal challenges—all with the goal of keeping 
them in school and on track to graduate. 


This report presents findings from MDRC’s randomized controlled trial evaluation of DPP at five Detroit 
community colleges. The campus coach and other DPP program components were well-implemented 
at four out of five of the colleges, and program participation was high. More than 90 percent of program 
group students responded to coaches’ initial outreach, and participation in coaching meetings remained 
high throughout the follow-up period for students enrolled in college. A student survey found that nearly 
90 percent of respondents rated the program as valuable or highly valuable. 


Many students continued to face significant obstacles to enrollment and persistence, however. Only 
about 65 percent of students in the program group enrolled in courses in the fall semester that they 
applied for a Promise scholarship. Of these enrolled students, more than 40 percent dropped out of 
school after one year. Students most often identified nonacademic barriers such as financial issues as 
the reason they dropped out of school. 


The evaluation found that more students in the DPP program stayed enrolled in school and earned 
more credits, compared with students who were offered the Promise scholarship alone. However, at 
the three-year mark, there was no evidence of an impact on degrees earned. 


It is clear that promoting college access is not enough. Programs must also tackle progress—helping 
students stay in school and get to graduation. The Detroit Promise scholarship combined with the Detroit 
Promise Path program supports is a step toward helping students stay in school. But there is still more 
to be done to help them get to graduation, too. 
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EXECUTIVE SUMMARY 


“My coach made going to school easy. | never really had someone to just listen and help me release my 
ideas and feelings. She is wonderful.” 


“My coach took time to get to know me, then discussed the program, financial aid.... He advised me on 
things | had no knowledge of.” 


“I deal with a lot of family issues and | can talk to my coach about anything.... He helped me figure out 
how to talk to my family and convince them that college is a stepping-stone to a better life. He lit the way 
for me in college to make something positive of myself.” 


— Three students in the Detroit Promise Path program 


Mn, of today’s college students—especially those in community colleges—are the first in 
their families to pursue higher education. Millions of undergraduates come from low-income 
families and battle self-doubt from within and stereotypes from without. Yet most community 
colleges are severely underfunded, and therefore are unable to provide the level of personal sup- 
port that these students need. For these and other reasons, the three-year graduation rate for 
first-time, full-time freshmen in community colleges is only 35 percent.’ 


But what if things were different? This report is the final publication from MDRC’s evaluation 
of the Detroit Promise Path (DPP) program. DPP is an evidence-based student services program 
for community college students. DPP was created by MDRC and the Detroit Regional Chamber 
in 2016 to help more Detroit high school graduates apply to and persist in college, accumulate 
credits, graduate, and potentially transfer to a four-year college. DPP builds on an existing three- 
year scholarship program called the Detroit Promise, which covers any gap between financial 
aid and tuition and fees for high school graduates to attend local community colleges. At the 
heart of DPP are campus coaches who help students acclimate to college, proactively reach out to 
them with help and reminders about tasks and deadlines, and offer a sympathetic ear to young 
people who may be grappling with personal challenges—all with the goal of keeping them in 
school and on track to graduate. 


This report presents findings from MDRC’s randomized controlled trial evaluation of DPP at 
five Detroit-area community colleges. About two-thirds of eligible students in 2016 and 2017 
were randomly assigned to be offered DPP (the program group), while the rest were assigned to 


1. National Center for Education Statistics, Integrated Postsecondary Education Data System, “Winter 2019- 
20, 200 Percent Graduation Rates component (provisional data)” (Washington, DC: U.S. Department of 
Education, 2020). Website: https://ies.ed.gov/. 
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receive the Detroit Promise scholarship alone (the control group). The main findings include 
the following: 


™ The coaching and other DPP program components were well implemented at four out of five 
colleges, and program participation was high. More than 90 percent of program group students 
responded to coaches’ initial outreach, and participation in coaching meetings remained high 
throughout the three-year follow-up period for students enrolled in college. 


= DPP was viewed favorably by program group students at all of the colleges. A student survey 
found that nearly 90 percent of respondents rated the program as valuable or highly valuable. 


= Many students continued to face substantial obstacles to enrollment and persistence. About 
65 percent of students who were offered the full DPP program enrolled in courses in the fall 
semester that they applied for a Promise scholarship. Of these enrolled students, more than 
40 percent dropped out of school after only one year. 


= Compared with students who were offered the Promise scholarship alone, more students in 
DPP stayed enrolled in school and earned more credits. But after three years, there was no 
evidence of an increase in degrees earned. 


™ Direct costs of DPP were $648 per student per year, for a total of $1,944 per student over the 
course of the three-year program. By enabling students to take more courses, the program 
added indirect costs of an additional $366 per student, which from the college perspective were 
at least partially offset by the increased revenue associated with students taking those courses. 


During the two study cohorts in 2016 and 2017, about 625 students per year received the Promise 
scholarship and of those students, about 400 students per year were offered the new Detroit 
Promise Path program.’ A total of 1,268 students are in the study. 


The Detroit Promise Path has been scaled up over the past three years to serve more students. 
Beginning in 2018, DPP expanded and now serves all incoming students at four of the five 
Detroit-area community colleges that participated in the study. This program shows that College 
Promise programs—a popular intervention aimed at improving college access by making tuition 
free—can be leveraged to address both college access and academic progress. 


2. The final semester of the program for the 2017 cohort was spring 2020, the same semester that the 
coronavirus pandemic hit Detroit. The city, and the students in the study, were highly impacted by the 
pandemic. The program quickly shifted to a fully remote version during this semester of the study period 
as well as subsequent semesters for later, non-study cohorts. Student need was drastically higher during 
this semester. 
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THE DETROIT PROMISE PATH PROGRAM MODEL 


Detroit’s community college students face steep odds. They must grapple with considerable 
institutional issues such as insufficient advising and counseling staff, high rates of contingent 
or adjunct faculty teaching introductory courses, and a confusing array of requirements and 
paperwork for financial aid and course selection. Community college students are referred 
to remedial courses at high rates, delaying their expected time of graduation. * They are often 
working while in school and cannot always prioritize studying.® For students from low-income 
households, the need to work is especially acute, as financial aid may cover tuition and fees but 
not transportation to school, child care, food, or other necessities. At the same time, students 
may receive enough aid for tuition but not enough to purchase all of their textbooks. 


Black students face additional hurdles such as self-doubt, a sense of not belonging, stereotyping 
from faculty or other students, and both subtle and overt racism. These challenges can be further 
amplified for students who are coming from predominantly Black high schools in Detroit to 
predominantly White colleges in the suburbs.’ And because community college students typi- 
cally spend little time on campus outside of class time—they are unlikely to live on campus, 
participate in clubs, or play sports—their emotional connection to college can be more tenuous, 
further reducing their sense of belonging in the college community compared with students at 
residential universities.® 


3. Jolanta Juszkiewicz, Trends in Community College Enrollment and Completion Data 2015 (Washington, 
DC: American Association of Community Colleges, 2015); Kevin J. Dougherty, Hana Lahr, and Vanessa 
S. Morest, Reforming the American Community College: Promising Changes and Their Challenges, 
CCRC Working Paper 98 (New York: Community College Research Center, Teachers College, Columbia 
University, 2017); Robert S. Feldman, The First Year of College: Research, Theory, and Practice on 
Improving the Student Experience and Increasing Retention (Cambridge, UK: Cambridge University Press, 
2017). 


4. Elizabeth Ganga, Amy Mazzariello, and Nikki Edgecombe, Developmental Education: An Introduction for 
Policymakers (New York: Community College Research Center, Teachers College, Columbia University, 
2018). 


5. Erin Dunlop Velez, Alexander Bentz, and Caren A. Arbeit, Working Before, During, and After Beginning at 
a Public 2-Year Institution (Washington, DC: National Center for Education Statistics, Institute of Education 
Sciences, U.S. Department of Education, 2018). 


6. Sandy Baum, Student Debt: Rhetoric and Realities of Higher Education Financing (New York: Palgrave 
Macmillan, 2016); Jennifer Ma, Sandy Baum, Pender Matea, and Meredith Welch, Trends in College Pricing 
(New York: The College Board, 2017). 


7. See Delila Owens, Krim Lacey, Glinda Rawls, and JoAnne Holbert-Quince, “First-Generation African- 
American Male College Students,” The Career Development Quarterly 58, 4 (2010): 291-300, and Shaun 
R. Harper and Isaiah Simmons, Black Students at Public Colleges and Universities (Los Angeles: USC 
Race and Equity Center, 2019) for more. In the DPP study, some of the colleges are predominantly Black 
institutions and some are predominantly White institutions. 


8. Regina Deil-Amen, “Socio-Academic Integrative Moments: Rethinking Academic and Social Integration 
Among Two-Year College Students in Career-Related Programs,” Journal of Higher Education 82, 1 (2011): 
54-91; Terrell L. Strayhorn, College Students’ Sense of Belonging: A Key to Educational Success for All 
Students (New York: Routledge, 2012). 
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The Detroit Promise, administered by the Detroit Regional Chamber and launched in 2013, is 
one of more than 300 free college or College Promise scholarships nationwide. These are often 
called “place-based scholarships,” in that they offer to cover college tuition and fees for all the 
students in a particular geographic area—in the case of the Detroit Promise, the entire city of 
Detroit. Like most College Promise scholarships, the Detroit Promise is available to all high school 
graduates and does not have merit-based eligibility criteria. Students may use their scholarship 
to attend any community college in the greater Detroit area. 


In the early years of Detroit Promise, Chamber staff members observed that the scholarship was 
helping more high school graduates enroll in college initially, but large numbers of scholarship 
recipients were dropping out before their second year. The Chamber wanted to incorporate 
student success components into the scholarship so Promise students would not only enroll in 
school but would be more likely to succeed there. As a result, MDRC and the Chamber created 
the Detroit Promise Path, which added four evidence-based service components to the existing 
Promise scholarship (shown in Figure ES.1). 


FIGURE ES.1 Detroit Promise Path Program Model 
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The heart of DPP is its campus coaching component. Students begin meeting with a coach in 
the late summer before their first semester of college. They are also offered a financial incentive 
to attend coaching meetings: a monthly gift card that is refilled with $50 each month they meet 
with their coach as directed. The money helps students pay for expenses not covered by financial 
aid. DPP lasts for the full three years of the Promise scholarship, including summer semesters, 
when students are encouraged to enroll in classes (paid for by the scholarship) or engage in a 
local summer jobs program called Grow Detroit’s Young Talent. DPP program operation is sup- 
ported by a management information system that coaches use to track participation in coaching 
sessions and to do outreach via email, phone, and text messages. 


The present study enrolled students in 2016 and 2017. Participants’ average age at study entry was 
18, as the Detroit Promise scholarship serves recent high school graduates. Students must enroll 
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in college within three semesters of high school graduation in order to access the scholarship, 
and most students enroll in the fall semester immediately following graduation. In the study, 
well over 90 percent of students identified as people of color, primarily as Black. Four out of five 
students reported that they did not live with a parent who had completed a bachelor’s degree. 


This evaluation of DPP aims to understand the program’s implementation, effects, and costs. 
The report first presents a descriptive analysis of the program’s implementation and academic 
outcomes for program students only. The findings are supported by evidence from interviews, 
focus groups, a student survey, program participation data, and college transcript records. Second, 
the report presents estimates of the program’s impact on student academic outcomes, using a 
randomized controlled trial (RCT) design, widely accepted as the gold standard of evaluation 
designs. In the RCT, students were randomly assigned to be eligible either for DPP (the program 
group) or for the Detroit Promise scholarship alone (the control group). Random assignment is 
a fair way to distribute limited spaces in a program, and it also allows unbiased estimation of 
the program’s impacts. The difference between the two groups represents the impact of the ad- 
ditional program components. This is not an evaluation of College Promise programs generally; 
the control group students in this study continued to receive the Detroit Promise scholarship. 


PROGRAM IMPLEMENTATION FINDINGS 


Overall, DPP was implemented with high quality and with high fidelity to the model at four of 
the five colleges. The service contrast, or the difference between what program group students 
received and what control group students received, was meaningful. Across cohorts and se- 
mesters, DPP students had positive views of the program, particularly their relationships with 
their coaches. In the words of one student, “My coach is a generous person. He is patient with 
me concerning my financial aid and he offers so much thoughtful advice with whatever issue 
may arise.” In a student survey, nearly 90 percent of respondents described DPP as valuable. 


Yet students reported that they continued to face significant barriers to success. Many struggled 
to afford basic needs; financial issues—whether academic, such as being able to afford textbooks, 
or nonacademic, such as being at risk of eviction and homelessness—were students’ most seri- 
ous concerns. 


These factors may underlie the high rates of students who did not enroll initially or who left 
college after a few semesters. Of the students who completed their Detroit Promise scholarship 
application during summer 2016 and summer 2017 and therefore entered the study, only about 
65 percent of them enrolled in courses the subsequent fall semester. This is much lower than 
seen in other MDRC community college studies. 


Impact Findings from the Randomized Controlled Trial 


This study prespecified three main or confirmatory outcomes: enrollment, credits earned, and 
degrees earned. As shown in Table ES.1 over the three years of the program, DPP helped more 
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TABLE ES.1 Three-Year Academic Outcomes Summary 


Program Control 
Outcome Group Group Difference P-Value 
Average number of semesters enrolled 2.9 2.5 0.4*** 0.002 
Average number of semesters enrolled, categorical* 
0 18.2 20.1 -1.9 0.417 
1-2 32.6 40.5 -7.9*** 0.005 
3-4 19.2 17.5 1.8 0.436 
5-6 30.0 21.9 8.1** 0.001 
Total credits earned 171 13.5 3.7°** 0.001 
Earned a credential (%) 7.2 6.8 0.4 0.771 
Sample size (total = 1,268) 829 439 


SOURCES: MDRC calculations using data from the National Student Clearinghouse and the Detroit Promise 
Path colleges. 


NOTES: Estimates are adjusted by site, interaction between race and gender, and ACT and SAT score. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 


Weights are calculated to make the effective (weighted) random assignment ratio the same in all random 
assignment blocks. The effective random assignment ratio is equal to the full sample’s random assignment ratio. 


Credential measures include three full academic years of data (in other words, fall, spring, and summer for three 
full years). Enrollment and credit measures include these same data except the final summer session (in other 
words, Year 3 summer is not available). 


aAverage number of semesters is the primary outcome in this analysis. The categorical version of this outcome is 
a complementary measure added to aid the interpretation of the primary measure. 


students make progress in higher education, based on the positive impacts on enrollment and 
credits earned, but it did not have a measurable effect on credential completion in this time- 
frame. Thirty percent of students in the program group enrolled in five or six semesters (out of 
six) compared with 21.9 percent of students in the control group, for an estimated impact of 8.1 
percentage points. Program group students earned more credits than the control group students, 
on average (17.1 credits compared with 13.5 credits, respectively), for an estimated impact of 3.7 
credits—a 27 percent increase. 


At the end of three years, 7.2 percent of the students in the program group earned a degree or 
certificate compared with 6.8 percent of students in the control group. The difference, 0.4 per- 
centage points, is neither practically nor statistically significant. While there is no measurable 
effect on credential completion at this time, it is possible that the impact on credit accumulation 
may lead to an impact on completion in the future. 
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Program Costs 


DPP cost an estimated $840 per student per year for a total of $2,520 per student over the three- 
year program—the college-perspective net cost. This includes direct costs, such as staff salaries 
and monthly student financial incentives, and indirect costs, such as the cost of additional 
credits taken by students in the program group compared with students in the control group, as 
well as increased revenue from those additional credits to the college. The total direct cost per 
program group student per year was $648. (Both the net and total direct costs include program 
group members who did not enroll.) Direct costs make up the bulk of the total cost of the pro- 
gram. More than half of the cost of the program came from the coaching component—namely, 
employing the campus coaches who worked directly with students. 


At this cost, the program helped more students persist in college and earn more credits. However, 
because DPP did not lead to more degrees earned at the three-year mark, it was not cost-effective 
for improving degree receipt. 


CONCLUSIONS 


Over the six semesters of the program, DPP helped more students make progress in college; they 
enrolled in more semesters and earned more credits. At this point, however, there is no evidence 
of an increase in degrees earned. MDRC hopes to secure funding for longer-term follow-up to 
continue to track these students. Around 30 percent remained enrolled in college during the 
final semester of follow-up. 


The Detroit Promise Path program was implemented well, and nearly all students contacted for 
the survey and qualitative study expressed a highly positive view of the program and of the help 
they received from their coaches. Yet these students continued to face great barriers to success. 
More research on how to address these issues is required. 


It is clear that college access is not enough. Programs must tackle both access—helping students 
get to college—and progress—helping them stay in school and get to graduation. The Detroit 
Promise Path program model is one way that College Promise and other free college programs 
can support students’ academic success. However, there is more to be done to improve gradu- 
ation rates as well. 
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Introduction 


Imagine you are getting ready to enroll in college for the first time. You’ve just 
graduated from high school—awesome. Now it’s August, and it’s a whole new 
world. You’re excited to learn new things in your classes, but are you supposed 

to pick College Algebra or Intro Statistics? You’]] have to figure out a new bus 
route that gets you to school early in the morning. Maybe you can look that up on 
your phone tomorrow when your data plan resets. You got an email about FAFSA 


verification—what the heck is that? Who should you ask? Okay, table that for today. 


Your parents are proud, but they’re also concerned. They never went to college 
and they need you to keep your part-time job to help with the rent. You can’t ask 
them what it will be like at school. Will other students want to be your friend even 
though you’re from the city and they’re from the suburbs? Will your professors 
make assumptions about you because of what you look like or where you come 
from? What if you have questions about where to go on campus—who should you 


ask? You don’t want people to think you don’t belong.... 


This is what starting college looks like for millions of students every year. Many of today’s in- 
coming freshmen—especially those in community college—come from low-income backgrounds 
and are the first in their families to pursue a degree. They often report battling self-doubt from 
within and stereotypes from without. Yet most community colleges are severely underfunded 
and unable to provide the kind of advising, counseling, and other supports that these students 
need. For this and other reasons, the three-year graduation rate for full-time, first-time fresh- 
men in community colleges is only 35 percent." 


But what if things were different? What if, before students even started college, someone was 


actively reaching out to them, texting them weekly, asking about their challenges and helping to 
solve them? Someone who understood where they came from and was there to help them succeed? 


1. National Center for Education Statistics (2020). 


“My coach made going to school easy. | never really had someone to just listen and help 
me release my ideas and feelings. She is wonderful.” 


“My coach took time to get to know me, then discussed the program, financial aid.... He 
advised me on things | had no knowledge of.” 


“| deal with a lot of family issues and | can talk to my coach about anything. He helped me 
figure out how to talk to my family and convince them that college is a stepping-stone to 
a better life. He lit the way for me in college to make something positive of myself.” 


—Three students in Detroit Promise Path 


This is the final publication from MDRC’s evaluation of the Detroit Promise Path (DPP) program. 
The evidence-based student services program was created by MDRC and the Detroit Regional 
Chamber in 2016 to help more Detroit high school graduates who apply to a two-year college 
persist in school, accumulate credits, graduate, and potentially transfer to a four-year program. 
DPP builds on an existing three-year scholarship program called the Detroit Promise, which 
covers tuition and fees for city high school graduates to attend local community colleges. At the 
heart of DPP are campus coaches who help students acclimate to college, proactively reach out 
with help and reminders about tasks and deadlines, and provide a sympathetic ear for young 
people who may be grappling with personal challenges—all with the goal of keeping them in 
school and on track to graduation. 


This report presents findings from MDRC’s randomized controlled trial (RCT) evaluation of 
DPP at five participating Detroit community colleges. About two-thirds of eligible students in 
2016 and 2017 were randomly assigned to receive the new DPP (the program group), while the 
rest were assigned to receive the new Detroit Promise scholarship alone (the control group). The 
main findings include the following: 


= The campus coach and other DPP components were well implemented at four out of five col- 
leges, and program participation was high. More than 90 percent of program group students 
responded to coaches’ initial outreach, and participation in coaching meetings remained high 
throughout the three-year follow-up period for students who enrolled in college. 


= DPP was viewed favorably by students at all five colleges. A student survey found that nearly 
90 percent of respondents rated the program as valuable or highly valuable. 


= Many students continued to face substantial obstacles to enrollment and persistence. Only 
about 65 percent of students who were offered the full DPP program enrolled in courses in 
the fall semester they applied for the Promise scholarship. Of these enrolled students, more 
than 40 percent dropped out of school after only one year. 
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= The evaluation found that, compared with students who were offered the Detroit Promise 
scholarship alone, more students in DPP stayed enrolled in school and earned more credits. 
But after three years, there was no evidence of an increase in degrees earned. 


™ Direct costs of DPP were $648 per student per year, for a total of $1,944 per student over the 
course of the three-year program. By enabling students to take more courses, the program 
added indirect costs of an additional $366, which from the college perspective was at least par- 
tially offset by the increased revenue associated with students taking those additional courses. 


This report presents three-year outcomes for study students covering the years 2016-2020. First, 
it describes the development of DPP, the program model, and the participating students. It also 
discusses the ramifications of the 2020 coronavirus pandemic for the city of Detroit, the stu- 
dents, the program, and the study. Next, the report shares qualitative and quantitative findings 
from the RCT evaluation, as well as an estimate of DPP’s cost and cost-effectiveness. Finally, the 
report presents important lessons for policymakers and practitioners who may be considering 
similar programs in their communities. 


WHAT IS THE DETROIT PROMISE SCHOLARSHIP? 


The Detroit Promise was launched in 2013 as the Detroit Scholarship Fund, to help more of the 
city’s high school graduates enroll in college. A “last-dollar” scholarship, it covers any gaps be- 
tween financial aid and tuition and fees that students may face. The scholarship is one of more 
than 300 free college or College Promise scholarships nationwide.” Like many College Promise 
scholarships across the country, Detroit Promise eligibility criteria are geographically specific: 
Students must have graduated from a Detroit high school, be a resident of the city of Detroit, 
and attend a college in the area. For the community college Detroit Promise scholarship, which is 
being studied here, there are no merit-based eligibility criteria; all local high school graduates 
may take advantage of it. (In 2017 the program expanded to help students with qualifying ACT 
or SAT scores and qualifying high school grade point averages pay tuition at public four-year 
colleges in Michigan.) 


The Detroit Promise scholarship is administered by the Detroit Regional Chamber as an edu- 
cational and workforce development program. Students can enroll in the program for up to 
one year after finishing high school, and they are eligible for scholarship dollars for a total of 
three years. Students are directed to enroll in college full time, though this requirement is not 
enforced, meaning students do not lose the scholarship if they drop below full-time status. The 
Detroit Regional Chamber and other community stakeholders regularly communicate to high 
school students citywide that the scholarship is there to alleviate their financial burden and to 
make sure they can afford to attend community college. 


2. The estimate comes from the Catalog of Local and State College Promise Programs, Fall 2020. The 
number of programs grows each year. See https://www.collegepromise.org/ 
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The Detroit Scholarship Fund operated for three years as a scholarship-only program. In those 
early years, Chamber staff members observed that while the scholarship seemed to help more 
high school graduates enroll in college, large numbers of recipients were dropping out of college 
before their second year. When staff members spoke to students about their experiences, they 
heard that students were facing myriad issues beyond financial aid that hindered their ability 
to reach their goal of a college degree. 


Many students said they were the first in their families to attend college and did not know how 
to navigate the campus environment. They needed assistance with financial aid paperwork, with 
choosing classes, and with improving their study skills, and they didn’t know whom on campus 
to ask. Many students also arrived at college academically underprepared; most of them were 
graduates of the Detroit public school system, which has gone through significant turmoil in 
the last few decades, including drastic declines in funding and enrollment due to what sociolo- 
gists call white flight.* The Detroit Public Schools Community District (DPSCD) has alternated 
between city control and state takeover in the past 20 years. 


The rate of students moving from one high school to another, particularly as low-enrollment 
schools have closed and new charter schools have opened, is higher in Detroit than in nearly any 
other city in the United States. High school graduation and college-going rates in the Detroit 
metro area are lower than national averages, and the city is also ranked lowest in the state and 
region.* While the DPSCD has stabilized in the past few years, the turmoil students in this study 
experienced during their K-12 years could have ripple effects throughout their college experience. 


For many of Detroit’s high school graduates, just getting to college is a victory. Yet students 
attending community colleges face the same kinds of systemic issues they experienced in their 
K-12 years. Across the board, community colleges receive less funding relative to what four-year 
colleges receive, despite serving so many students who need additional support and guidance 
to succeed. This basic inequity in college funding between open-access, two-year colleges and 
four-year colleges serving better prepared, higher-income students is a nationwide issue. 


It’s not surprising, then, that community college students face steep odds. Nationwide, only 
about 20 percent of first-time, full-time community college students graduate within two years 
of starting; just over one-third graduate after three years.° At the schools that Detroit Promise 
students attend, the completion rates are even lower, ranging from 2 to 8 percent completion at 
two years and 6 to 20 percent completion at three years. Add to that the fact that most students 
who receive Detroit Promise scholarships come from low-income families and are receiving 
federal Pell Grants, and the graduation rates are lower still. Students must also grapple with 
many institutional issues, including lack of sufficient advising and counseling staff, high rates 


3. Asummary of the history of Detroit public schools can be found at https://landgrid.com/reports/ 
schools#what-happend. 


4. For more information, see the Detroit Regional Chamber’s State of Education annual report. The 2019- 
2020 report can be found at https://issuu.com/detroitregionalchamber/docs/soe book web. 


5. National Center for Education Statistics (2020). 
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of contingent or adjunct faculty teaching introductory courses, and a confusing array of require- 
ments and paperwork for financial aid and course selection.® 


Other systemic issues may present additional barriers. For example, research shows that Black 
students and students from low-income households (again, the majority of Detroit Promise 
scholarship recipients), are more likely than White students to be selected for Free Application 
for Federal Student Aid (FAFSA)’ verification, an often difficult process in which students may 
experience delays in receiving Pell grants, or may lose out on a semester of financial aid altogether.® 


Community college students are also referred to remedial courses at high rates, delaying their 
expected time of graduation.? They are commonly working while in school, meaning they can- 
not always prioritize studying.'® For students from low-income households, this is especially 
acute, as financial aid may cover tuition and fees but not textbook costs. Nor does it cover the 
cost of transportation back and forth to school, child care, food, or other necessities." Black 
students often face additional hurdles, such as self-doubt, a reduced sense of belonging, stereo- 
typing by faculty or other students, and both subtle and overt racism. These challenges can be 
further amplified for students who are coming from predominantly Black schools in Detroit to 
predominantly White colleges in the suburbs.'’? And because community college students often 
spend little time on campus outside of class time—they are unlikely to live on campus, partici- 
pate in clubs, or play on school sports teams—their social connection to the school can be more 
tenuous, further reducing their sense of belonging in the campus community."* 


THE DETROIT PROMISE PATH PROGRAM MODEL 


Seeing the struggles their students were having, the Detroit Regional Chamber decided to in- 
corporate additional evidence-based program components into the Detroit Promise scholarship. 
To do this, the Chamber and MDRC partnered to create DPP, which adds four components to 
the existing scholarship program. The program model is shown in Figure 1.1. 


The core element of the program is its coaching component. DPP students begin meeting with 
their coaches in the late summer before their first semester of college. Coaches use a proactive 
and holistic approach: They actively reach out to students using multiple modes—text messages, 


6. Dougherty, Lahr, and Morest (2017); Juszkiewicz (2015); Feldman (2017). 

7. FAFSA is the form used to apply for Pell Grants and other need-based federal financial aid. 
8. Holzman and Hanson (2020). 

9. Ganga, Mazzariello, and Edgecomb (2018). 

10. Velez, Bentz, and Arbeit (2018). 

11. Baum (2016); Ma, Baum, Matea, and Welch (2017). 


12. See Owens, Lacey, Rawls, and Holbert-Quince (2010) and Harper and Simmons (2019) for more. In the 
Detroit Promise Path evaluation, some of the colleges are predominantly Black institutions and some are 
predominantly White institutions. 


13. Deil-Amen (2011); Strayhorn (2012). 
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FIGURE 1.1 Detroit Promise Path Program Model 


SCHOLARSHIP OTHER SERVICES AND BENEFITS 


Covers Offers Offers $50 Engages Uses a 
tuition campus each month students in management 
and fees coaching for expenses the summer information 
after + and requires not covered by by encouraging system 
financial aid each student financial aid, enrollment or to track student 
for students to meet with such as bus connecting participation 
enrolled full time a coach twice passes or them them to and automate 
and assists per month books, to local initiatives appointments 
them with students who such as and financial 
summer meet with summer job incentives 
tuition coaches programs for better 
monitoring 


email, and phone calls—to complement in-person, one-on-one meetings. Unlike traditional 
drop-in advising, the DPP model puts the onus on coaches to regularly send students messages 
and reminders. They talk with students about a wide variety of topics, including students’ ex- 
periences in school, time management, study skills, problem-solving strategies, personal issues, 
managing competing responsibilities, and building a sense of self-efficacy. Coaches also help with 
questions such as how to pick a major and they connect students with advisors or counselors for 
assistance in selecting courses or mapping out a graduation plan. One coach described the role 
as being a conduit: “If a student connects to me, I will connect them to advisors, financial aid, 
the food bank—everything they need.” Most DPP students are required to have two in-person 
meetings with their coach every month. Some students who are excelling in school are given 
the option to replace one of those meetings with a group meeting, which allows them to con- 
nect with other students in the program. To ensure that coaches are able to meet student needs, 
DPP uses a case management model in which students always see the same coach, and caseload 
ratios are kept under 1:150—far below national averages." 


Students are offered a financial incentive in the form of a gift card that is refilled with $50 for 
each month that they meet with their coach. This incentive helps students pay expenses not 
covered by financial aid. The DPP program lasts for the full three years of the Promise scholar- 
ship, including summer semesters, when students are encouraged to either enroll in classes (also 
paid for by the scholarship) or join a local summer jobs program called Grow Detroit’s Young 
Talent. The goal is to keep students engaged in some productive activity over the summer so 
they stay connected to their school and the DPP program, and therefore may be more likely to 
return to college in the fall. The entire DPP program operation is supported by a management 
information system (MIS) that coaches use to track participation in coaching meetings and for 


14. DPP caseloads were always under 1:150 but usually hovered around the 1:125 range. For more about 
national statistics on caseloads, see https://nacada.ksu.edu/Resources/Clearinghouse/View-Articles/ 
Advisor-Load.aspx. 
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email, phone outreach, and text messaging with students. The texts incorporate several behav- 
ioral science strategies to make messages more salient to students and easier for them to act 
on; for instance, the texts may include implementation prompts, clear instructions, and regular 
reminders about tasks and deadlines. 


DPP is designed to meet the needs of local students by adding relevant, evidence-based student 
supports to the existing Detroit Promise scholarship. Multiple experimental studies have shown 
that approaches such as enhanced advising and financial incentives can have positive, modest 
effects.'° However, evaluations of programs that combine multiple evidence-based components 
and provide services to students for a longer period of time have shown larger effects.'® The 
Chamber and MDRC also customized program components based on conversations with students 
about what would work best for them. For example, most students preferred texting over email, 
so the MIS has a robust text messaging platform. Unlike many multifaceted community college 
programs that have been evaluated by MDRC and other researchers, management of the DPP 
program takes place outside of the colleges; while some coaches are college employees, most are 
employees of the Detroit Promise and are supervised by the Chamber. This is a function of how 
the Detroit Promise operated prior to the creation of DPP. 


MDEC provided extensive technical assistance during the first two years of DPP to set up the 
program, develop cost-effective management strategies, teach staff members how to use the 
MIS and train them in behavioral science messaging strategies, and ensure the program was 
operating as intended. This study includes students attending one of five Detroit-area com- 
munity colleges: Henry Ford College, Macomb Community College, Oakland Community 
College, Schoolcraft College, and Wayne County Community College District. All of them are 
open-access, two-year schools. 


EVALUATION STRATEGY 


This evaluation aims to understand DPP’s implementation, effects, and costs. First the report 
presents a descriptive analysis of the program’s implementation and students’ academic outcomes, 
for program students only. The findings are supported by evidence from interviews, focus groups, 
a student survey, program participation data, and college transcript records. 


Second, the report presents estimates of the program’s impact on student academic outcomes 
using a RCT design, widely considered to be the gold standard in social science evaluation re- 
search. In the RCT, students were randomly assigned to be eligible either for the new DPP pro- 
gram (the program group) or for the scholarship alone (the control group). Random assignment 
is a fair way to distribute limited spaces in a program, and it also allows unbiased estimation of 


15. Bettinger and Baker (2014); Mayer, Patel, and Gutierrez (2015); Page, Castleman, and Meyer (2020); 
Scrivener and Coghlan (2011); Welbeck, Diamond, Mayer, and Richburg-Hayes (2014); Welbeck, Ware, 
Cerna, and Valenzuela (2014). 


16. Barr and Castleman (2017); Carrell and Sacerdote (2017); Evans, Kearney, Perry, and Sullivan (2017); Page, 
Castleman, Kehoe, and Sahadewo (2017; Rolston, Copson, and Gardiner (2017); Scrivener et al. (2015); 
Sommo, Cullinan, and Manno (2018). 
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the program’s impacts. The difference in outcomes between the program group and the control 
group represents the impact, or value-added, of the additional program components. In this 
study, the impact measured is that of the new components, not the scholarship itself. 


The report’s experimental estimate of the program’s impact used three sources of information: 


Demographic data. Before random assignment, students completed the Detroit Promise scholarship 
application on the Chamber’s website. The application contained questions about demographics 
and other background information. The data were used to describe the study sample, to docu- 
ment that the characteristics of the program group members and the control group members 
were similar at the outset of the study, and to define subgroups of interest. 


College records. Measures of academic outcomes were obtained from college transcripts, which 
were provided to MDRC by the five participating colleges.” 


National Student Clearinghouse. These data, which the Clearinghouse gathers from nearly all 
postsecondary institutions in the United States, were used to examine academic outcomes such 
as enrollment, transfer, and graduation rates. 


STUDY ENROLLMENT 


Study enrollment took place in summer 2016 and summer 2017, ahead of the fall 2016 and fall 
2017 semesters, respectively. A total of 1,268 students were in the study. The RCT was conducted 
by MDRC; the qualitative study was conducted by MDRC and the Youth Policy Lab at the 
University of Michigan (YPL). 


All students who were eligible for the Detroit Promise scholarship were also eligible for the 
new Detroit Promise Path program. For the RCT, eligible students were randomly assigned to 
either a program group, in which campus coaches made contact with students and students were 
eligible for the additional financial incentives; or a control group, in which students continued 
to receive their Promise scholarships and regular college services, but did not receive outreach 
from coaches, messaging, or monthly financial incentives. 


Students were notified of the DPP program and study and were given the option to decline to 
participate; no students chose to decline the program and study.'® At the time that they com- 
pleted the scholarship application and joined the study, students were asked which of the five 
participating community colleges they would be attending. Following random assignment into 


17. Data were collected through the summer 2020 semester. However, due to the coronavirus pandemic, there 
may be some missing data in spring/summer 2020. 


18. It is possible that some students, upon reading the informed consent language in the application, chose 
not to complete the application at all. The researchers do not have a way to measure this. Anecdotally, 
program staff did not believe this was an issue. 
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the program group, the DPP students were cold called by coaches inviting them to the new 
program and asking them to schedule their first one-on-one meeting. 


To be clear, this study is an evaluation of the impact of the additional services offered in DPP, 
on top of the Detroit Promise scholarship; it is not a test of Promise scholarships generally. 
Students in the program group continued to be eligible for the Detroit Promise scholarship, as 
did control group students, who continued to receive Promise scholarship dollars as they would 
have in the absence of the study.'® 


An additional note on terminology: While the colleges in the study refer to the semester begin- 
ning in January and ending in May as the “winter semester,” this report calls this semester the 
“spring semester” to match terminology used in most colleges and other MDRC studies. 


STUDENT POPULATION 


Table 1.1 presents the demographic characteristics of the students in the study. The average age 
at study entry was 18, as the Detroit Promise scholarship program serves recent high school 
graduates. Students must enroll in college within three semesters of high school graduation in 
order to access the Promise scholarship, and most students enroll in the fall semester immedi- 
ately following high school graduation. In the study, well over 90 percent of students identified 
as people of color, primarily as Black. Four out of five students reported that they did not live 
with a parent who had completed a bachelor’s degree.”° 


The federal government reports that nearly half of Detroit residents under the age of 18 live in 
poverty, while the United Way puts that number at more than 70 percent.”' At the five participating 
colleges in this study, rates of federal Pell Grant receipt for first-time, full-time students ranged 
from 27 percent to 81 percent, with the highest percentages at the two colleges enrolling the greatest 
numbers of Detroit Promise scholarship recipients. Detroit has experienced generation after gen- 


19. Most Detroit Promise students qualify for federal, need-based Pell Grants that fully cover the cost of tuition 
and fees at Detroit-area community colleges. As a result, most students were not receiving substantial 
scholarship dollars, as their tuition and fees were already covered. However, all students, regardless 
of treatment group, were able to get the scholarship dollars to which they were entitled. Proponents of 
College Promise scholarship programs argue that simply offering the scholarship may make students more 
likely to enroll. since many students from low-income households do not realize that they are eligible for 
need-based financial aid. The idea is that even for students who do not receive any scholarship dollars, the 
“promise” of free tuition may relieve their financial anxiety and make them believe college is financially in 
reach. 


20. Most students in the study were the first in their families to attend college. The Detroit Promise application 
asks students whether they live with a parent who has completed a four-year college degree, which is 
slightly different from the general definition used by researchers of “first in the family to attend college.” 
However, in the study survey (reported below), students were asked whether they had any family members 
who had completed any college degree (a broader question than in the Detroit Promise application). In the 
survey, only about one-third of students reported having any family member who had completed a degree, 
while one-quarter had relatives who had attended college and not finished, and 20 percent had no family 
members who had been to college at all. 


21. United Ways of Michigan (2017); U.S. Census Bureau (2018). 


Motor City Momentum | 9 


eration of racial disparities and systemic issues in many realms, from education and employment 
to housing and health, leaving many residents—especially Black residents—in poverty.”? Detroit’s 
Black students have also been historically marginalized and therefore historically underrepresented 
in higher education enrollment and completion. In short, Detroit Promise Path serves a student 
population that has faced significant barriers and needs additional support to improve outcomes 
and reduce disparities. 


TABLE 1.1 Selected Baseline Characteristics of Program and Control Group 


Members 
Program Control 
Characteristics (%) Group Group Difference P-Value 
Female 58.3 60.6 -2.3 0.432 
Male 41.7 39.4 2.3 0.432 
Age 17.9 17.9 0.0 0.944 
Race 
Black or African-American 80.3 80.6 -0.4 0.880 
Hispanic or Latino? 12.6 11.0 1.5 0.432 
Two or more races 4.3 3.8 0.5 0.675 
Asian, Native Hawaiian, or other Pacific 
Islander 1.7 21 -0.5 0.539 
White 1.2 1.6 -0.4 0.546 
American Indian or Alaska Native 0.0 0.8 -0.8** 0.012 


Does not live with a parent who has earned a 
bachelor’s degree 81.1 77.8 3.3 0.165 


Sample size (total = 1,268) 829 439 
SOURCES: MDRC calculations from Detroit Promise scholarship application data. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Missing values are only shown for items with more than 5 percent missing values. 

Weights are calculated to make the effective (weighted) random assignment ratio the same in all 
random assignment blocks. The effective random assignment ratio is equal to the full sample’s random 
assignment ratio. 

SAT and ACT scores are not included because more than half of all test score data is missing. SAT 
score was added to the Detroit Promise scholarship application in the second year of the study, at which 
time both test score questions became optional. For students reporting a score, there are no statistically 
significant differences between treatment groups (p=0.42 for SAT, p=0.17 for ACT). 

aHispanic or Latino students may be of any race. 


22.For a summary of the many issues, including racial discrimination in housing, employment, and schooling, 
see Sugrue (1996, revised 2014). For examples across policy domains, see Orfield and Lee (2005); Schulz 
et al. (2002); Chetty, Hendren, Kline, and Saez (2014); Darden and Same (2000); Detroit Regional Chamber 
(2020). 
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Program Implementation and Student 
Participation 


This chapter describes the implementation of the Detroit Promise Path program and students’ 
experiences in it, using qualitative research, a student survey, and an analysis of program par- 
ticipation. The chapter focuses on students in the program group. The next chapter compares 
academic outcomes for program group students and control group students. 


PROGRAM IMPLEMENTATION 


MDERC and the Youth Policy Lab (YPL) conducted two rounds of qualitative implementation 
research. This included interviews and focus groups with program staff, interviews and an on- 
line survey with students, and a review of program participation data in the DPP management 
information system (MIS). 


Overall, the program was implemented with high quality and high fidelity to the model, at four of 
the five participating colleges. All of the DPP components were made available to students in the 
program group.’ The service contrast, or the difference between what program group students 
received and what control group students received, appeared to be quite high at most colleges 
in most semesters. At one college, the program operation was drastically different as a result of 
staff turnover and a lack of support for the program from senior college leadership. Program 
implementation and fidelity are discussed below for the four stronger implementing colleges 
first, followed by a description of the challenges at the fifth college. 


1. Researchers might assess fidelity to a program model as the extent to which the planned program 
services are offered or made available to students. An alternative is to examine the extent to which planned 
program services are received or experienced by students. In this study, researchers used the former 
and assessed fidelity based on whether program components were implemented and made available to 
students as designed, since this is within the control of the program implementers. See Weiss, Bloom, and 
Brock (2014). 
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Campus coaching began almost immediately after students joined the study during the summer 
and early fall of 2016 and 2017. Coaches reached out to students assigned to the program group by 
emails, text messages, and phone calls, and informed them of the benefits of the new program. 
Students were encouraged to schedule their first in-person coaching meeting as soon as possible, 
with the goal of meeting before school started or during the first week of the fall semester. More 
than 90 percent of students responded to outreach from coaches. As planned, coaches’ caseloads 
of active students remained below the program model’s 1:150 goal throughout the study period. 


Coaches continued to reach out to students via telephone, text message, and email throughout 
the semester, with the goal of building relationships and identifying student needs early. They 
also continued to reach out to students who were not responsive,” and were able to reengage some 
of them well into the semester or even the next semester after months of silence, often when a 
student was experiencing a problem. MDRC’s Center for Applied Behavioral Science (CABS) 
provided technical assistance to coaches so they could learn how to use behavioral science prin- 
ciples to inform the content of the text messages they sent to students. For instance, the messages 
used clear language and implementation prompts to help students complete complex tasks, such 
as filling out the Free Application for Federal Student Aid (FAFSA) form. Text messages tied to 
milestones such as registration also included task-oriented implementation prompts. Coaches 
identified which times of day got the best response rates and ensured that future messages 
were sent at those times. Across the program years, the messages were consistently sent out as 
planned. Service contrast for the coaching component was high. In contrast with the experi- 
ence of students in the program group, students in the control group did not receive proactive 
outreach from coaches or regular text messages with reminders or prompts to complete tasks. 


In interviews and focus groups, nearly all of the students said that coaching was the most im- 
portant component of the DPP program. Their assessment of their coaches and the support they 
received from them was also almost universally positive. Students regularly used words such 
as “helpful,” “generous,” “caring,” and “motivational” to describe their coaches. One student 
said, “My coach keeps me on track and makes sure I’m doing exactly what I should be doing. 
If I have any problems or bumps, he helps me find a solution to fix it.” Another student said, 
“[Coaching] was very helpful because I can tell my coach anything, and they helped me with 
personal stuff, even though it wasn’t in their job to do so.” Another student echoed those senti- 
ments: “[My coach] helped me balance my personal life and education and gave me great tips to 
stay organized and improve my study habits.” Students appreciated that the coaches understood 
their experiences, often because the coaches came from similar backgrounds and were from 
the city of Detroit. Students also noted the value of having coaches who had attended the same 
community colleges where they coached. One student said the DPP coach was “the best African 
American educational role model I have had the pleasure to be introduced to.” 


The financial incentive to attend coaching meetings was also implemented as planned. Students 


automatically received $50 ona refillable gift card each month, contingent on meeting with coaches 
as directed. The implementation of this component was more challenging for the Chamber than 


2. The exception was if a student asked not to be contacted again. If, instead, students simply did not 
respond, coaches continued to send them messages periodically to try to reengage them. 
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originally foreseen, however. There were more logistical hurdles than anticipated, including a 
longer turnaround time for account processing—as much as two weeks—and some unexpected 
issues with vendors for students who did not have or did not provide Social Security numbers or 
bank account information. The process to replace lost gift cards was also more time-consuming 
than expected and required more staff involvement than desired. While some of these logistical 
issues were eventually smoothed out, it was not possible to shorten the two-week turnaround time 
to fill the cards each month. Regardless, the monthly incentive represented a unique aspect of 
DPP: Researchers could not identify an analogous program offering flexible financial supports 
at any of the colleges. Students said they greatly appreciated the gift cards (though some wished 
that the monthly refill occurred earlier in the month). Most often, students in focus groups said 
they used the gift cards to buy monthly bus passes or to purchase food on campus. This finding 
was mirrored in the student survey results (discussed below). 


The summer engagement component of DPP was also implemented as designed. Each spring, 
coaches set aside specific meetings, typically in March or April, during which students were 
asked to decide how they would spend their summer and to make a plan. Coaches recommended 
that they take summer courses. Students might also opt to participate in a local summer jobs 
program called Grow Detroit’s Young Talent. Not all students followed through on their plans, 
however, and many students who chose to participate in the jobs program reported having dif- 
ficulty signing up for it or getting through its application process. 


A notable contextual change occurred midway through the study. As described earlier, the 
Detroit Promise scholarship also covers students’ summer tuition. This was true for students 
in both the program group and the control group; however, only program group students re- 
ceived strong messaging from coaches about the importance of taking summer courses. In the 
first year of DPP, that messaging led to large impacts on program group students’ likelihood of 
enrolling in summer classes. However, following the reinstatement of year-round Pell Grants 
(also called Summer Pell) in 2018, the context changed greatly. The colleges began messaging 
to all Pell-eligible students—which nearly all study students were—that they now had federal 
financial aid to cover summer courses. As a result, in later years of the study, summer enrollment 
rates increased among students in the control group and DPP no longer effected a large impact 
because the rates were similar for both groups of students. (Summer enrollment is examined in 
more detail in the next chapter.) 


The final DPP program component, the MIS, was implemented with high fidelity. Program staff 
used the MIS daily to track students’ meetings with their coaches; to make notes about follow- 
up items; to identify students who had or had not completed requirements such as attending 
meetings or completing FAFSA; and perhaps, most often, to communicate with students via 
text messaging using a computer-based platform that plugged into the MIS. Program leader- 
ship also used the MIS to run regular reports about how DPP was operating and to identify 
areas for improvement. For example, if a report showed that a particular group of students 
or a coaching caseload was not participating in equal rates, the staff could allocate additional 
support to improve participation. 
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At one college, implementation of the program varied from year to year; ultimately college leader- 
ship and the Chamber decided to move the program entirely off campus. Three components—the 
financial incentive, encouragement to enroll in summer courses, and the use of the MIS—closely 
resembled what was used at the other colleges (and the program model); however, the coaching 
component went through several iterations over time. Initially, the coaches were employed by 
the college and had an office on campus. Following coaching staff turnover, however, and the 
college leadership’s growing skepticism about the program, this school limited DPP to just one 
of its campuses for a semester, and then directed the coaches to hold meetings off campus. As 
a result, DPP students met with coaches in community locations or by video chat. The lack of 
support from college leadership presented a number of issues for the program that worsened 
over time. These included a diminished ability to solve financial aid issues, a lack of access to 
student data such as enrollment, and the inability of the coach to meet students on other cam- 
puses, among other issues. As a result, the coaching component at this college was drastically 
different compared with the other colleges and did not adhere to the program model. The service 
contrast following the shift to an off-campus version of the program is unclear; researchers were 
unable to gather the same types of data to assess the control condition at this college after 2017. 
Notably, this college enrolled the second-largest number of students in the MDRC study. The 
implications of how the program’s implementation at this college might have affected students’ 
academic outcomes is discussed in the Impacts section below. 


CONTINUOUS IMPROVEMENT 


MDRC and the Detroit Regional Chamber used a continuous improvement framework to review 
DPP program operation on a regular basis, to identify areas for improvement, and to continue 
to tweak components to meet student needs. One of the improvements included creating Excel 
tools to predict future caseloads and staffing needs. This allowed DPP staff to plan carefully 
how to staff the program as it grew while ensuring that caseloads remained low enough to meet 
student needs. Program staff were able to have important conversations about whether the 1:150 
caseload ratio worked for all participating colleges, and whether students from particular schools 
or areas of the city had greater needs and fewer external resources and required more support 
from the program. This has proven beneficial as the program has scaled up, too. See Box 2.1 for 
more information on the program’s scale-up efforts. 


DPP staff also learned lessons about and improved operations for student messaging. The program 
uses a two-way text messaging platform that plugs into the MIS to communicate with students. In 
early semesters, program staff experimented with automating the messages they sent out. They 
quickly found that if they weren't available to read and reply to students’ text message responses 
immediately, students were discouraged and assumed it was a robot texting them—especially 
early on, before they had formed strong relationships with their coaches. Timing the automated 
messages so that staff members were on hand to reply promptly made a big difference in student 
response rates. Timing the messages to when students were between classes, especially at lunch 
time, also helped improve response rates. 


141 Motor City Momentum 


Box 2.1 


Reaching More Students: Scaling Up Detroit Promise Path 


One of the notable outcomes of the Detroit Promise Path (DPP) study was that promising early 
findings led the Detroit Regional Chamber and other city stakeholders to expand the program 
before the study was completed. 


DPP builds on the Chamber’s existing three-year scholarship called the Detroit Promise, which 
covers tuition and fees for high school graduates to attend local community colleges. For the 
MDRC study, about two-thirds of eligible students in 2016 and 2017 were randomly assigned 

to receive full DPP coaching support and other services (the program group), and the rest were 
randomly assigned to receive the Promise scholarship alone (the control group). This was done to 
allot limited spaces fairly, given funding constraints, as well as to allow for a rigorous evaluation of 
the new program. 


But early findings showed statistically significant improvements in enrollment, full time enrollment, 
and credit accumulation—all important intermediate outcomes. So in 2018, the Chamber secured 
additional funding to begin expanding DPP. Based on coaching caseloads and the colleges’ level 
of support for the program, DPP was able to offer its services to al/ incoming Detroit Promise 
students at four of the five participating community colleges that fall, not just to students in 

the study. In 2019, the Chamber added a sixth community college, Jackson College, to the 
scholarship program, and hopes to add a coach after the pandemic is over. 


HOW DID THEY DO IT? 


The Chamber used the early research findings to show funders and local stakeholders how 

well the program was working. Local philanthropies, especially those engaged in economic 
development and education initiatives such as Detroit Drives Degrees, which focuses on 
postsecondary attainment, were eager to support a program proven to improve students’ 
persistence rates. The Chamber also used a great resource: the voices of DPP students 
themselves. Giving students the opportunity to talk about the value of the program and to tell 
their personal stories of success in college helped make the case that this was a program worth 
expanding and sustaining long-term. 


WHAT’S NEXT? 


Before the 2020 coronavirus pandemic, the Chamber’s goal was to continue to scale up the 
program by adding a coaching component for Detroit students who used the Promise scholarship 
to attend area four-year colleges. While these plans are temporarily on hold, accommodating 
students who start off at four-year colleges as well as students at two-year schools who are 
transferring into baccalaureate programs is a priority for the Chamber as well as for stakeholders 
in Detroit Drives Degrees. At the same time, the state of Michigan has created a statewide tuition 
support program called Michigan Reconnect. As the program grows, one policy goal is to develop 
supports based on lessons from Detroit Promise Path and other coaching programs. 


An open question based on the qualitative research is how to help Detroit Promise students 
who do not enroll in school. About one-third of students who signed up for the Detroit Promise 
scholarship and entered the study in summer 2016 and summer 2017 did not enroll in college 
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the following fall semester. While researchers were unable to get comprehensive information on 
these students’ experiences, qualitative interviews with some of these students as well as reports 
from DPP staff who contacted the program group students who did not enroll, identified one 
common issue: financial concerns. Even with the Promise scholarship in place, students often 
experienced issues with either FAFSA completion or FAFSA verification that made them un- 
able to enroll in courses or caused them to be dropped from courses due to nonpayment. Many 
students also reported that they could not afford other expenses not covered by financial aid, 
such as transportation, rent, or child care, and opted to work instead of attending school. As 
noted earlier, research has shown that Black students are more likely than White students to be 
selected for FAFSA verification, as are students from low-income households compared with 
students from higher-income households; DPP coaches saw FAFSA completion and verification 
as the greatest systemic issue for students. Financial aid issues were also seen as a significant 
driver of “summer melt,” in which students who intended to enroll in college in the fall semester 
disengaged during the summer and did not enroll after all. 


In fact, financial issues loomed large for most DPP students. Said one, “I was not having the 
resources I need for class on time due to financial issues. Maybe next semester I will.” The “op- 
portunity cost” of attending college was also a concern. Many students, enrolled or not, worked 
full or part time and said their families relied on their earnings to help pay for rent and food 
for the household. Other issues typically associated with summer melt, such as losing interest 
in college, or changes in circumstance, such as joining the military or moving away, were rare 
compared with financial struggles, coaches said. 


Another challenge might be categorized as what one student called “school skills.” Many stu- 
dents reported that they struggled with time management, note-taking and study skills, and 
had trouble paying attention in class when things moved at a faster pace than they had been 
used to in high school. Although DPP created additional resources and actively helped students 
develop these skills, some students said the deluge of difficulties made it hard to stay focused 
and motivated; others concluded that college was not really for them and opted to work instead. 


ACADEMIC OUTCOMES FOR DETROIT PROMISE PATH 
STUDENTS 


As previously described, this evaluation includes two cohorts of students who fully completed 
the Detroit Promise scholarship process. In the randomized controlled trial (RCT), a total of 829 
students were randomly assigned to be offered DPP (the program group), and 439 were assigned 
to receive the Detroit Promise scholarship alone (the control group). Completing the scholar- 
ship process included registering for college and getting a college ID number. Therefore, it was 
expected that most students would be enrolled in college in Semester 1, although students had 
up to three semesters after high school graduation to begin taking up the scholarship. However, 
only 65 percent of program group students enrolled in courses in Semester 1—notably lower than 
expected. Some of these students enrolled in subsequent semesters. The overall percentage of 
students who ever enrolled increased to 76 percent as of Semester 2 and 79 percent as of Semester 
3. As noted earlier, qualitative research suggests that the most common reason for delayed en- 
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rollment was FAFSA issues, especially for students who were flagged for FAFSA verification and 
were not able to get their application resolved before the semester started. 


How did the students in the program group who began college in Semester 1 fare? Figure 2.1 
presents enrollment and credentials earned for this subpopulation over six semesters. Eighty 
percent of the students who were enrolled in Semester 1 persisted into the second semester and 
56 percent of them were still enrolled in Semester 3 the following fall. On average, throughout 
the three-year follow-up period, the students who began in Semester 1 enrolled in 3.7 semesters 
(out of a possible 6), and earned 23 credits (not shown in table). 


FIGURE 2.1 Persistence and Graduation Rates Among DPP Program Group Students Who 


Enrolled in Semester 1 
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SOURCE: MDRC calculations using data from the National Student Clearinghouse and the Detroit Promise Path colleges. 


NOTES: This figure displays the enrollment over time for the 542 program group students who enrolled in Semester 1. 


According to federal Integrated Postsecondary Education Data System (IPEDS) data from 2016, 
the three-year graduation rates for Pell-eligible students at the five DPP colleges ranged from 
5 percent to 14 percent, with a weighted average of 7.2 percent.* Looking at the DPP students 
who enrolled in Semester 1 (also shown in Figure 2.1), 10.7 percent earned a degree or certificate 
in this timeframe. A comparison of the two data sets likely would not reveal much about the 
effectiveness of Detroit Promise Path, however. For one thing, the populations are not identi- 
cal: Participation in DPP is limited to Detroit residents only, most of whom are Pell-eligible, 
whereas the IPEDS numbers include all Pell-eligible students who attended these five colleges. 


3. The weighted average is based on the proportion of students in the sample coming from each of these 
institutions. 
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The time period is also not the same: In the study, DPP graduation rates are from 2019 and 
2020, and the IPEDS rates are from 2016. However, readers may find it useful to consider how 
the DPP students performed relative to the schoolwide numbers that are available. The DPP 
students who began college in Semester 2 or 3 fared similarly to the Pell-eligible students, with 
somewhat lower levels of credit accumulation and degree completion, given that they started 
college a semester or two later. 


STUDENT PARTICIPATION 


Students in the program group were directed to meet with their coach one-on-one twice per 
month in their first semester in college as well as in any semester thereafter if they were strug- 
gling, either academically or personally. Students who were doing well could replace one of 
their monthly one-on-one meetings with a group session or group activity with the coach. One 
student described the group sessions as “cool” because “you get to meet other students who are 
from the same place as you and making the same progress in college.” 


Figure 2.2 presents student participation in coaching, derived from MIS data. (As incentive re- 
ceipt is tied directly to coaching participation, it is not presented independently.) Throughout 
the study period, most enrolled students met with their coach five or more times per semester, 
on average. While this is slightly below the average required to fully meet the twice-per-month 
requirement (the number of meetings varied according to the academic calendar each semester), 
it is encouraging and shows a high level of engagement in the program among enrolled students. 
In most semesters, student participation rates increased over the course of the semester (not 
shown). For instance, students who did not respond to outreach in August or September might 
begin to respond and attend coaching meetings in October. 


For the final study cohort, participation rates remained steady in spring 2020, during the COVID-19 
pandemic. Note that during the pandemic, the in-person coaching meetings were replaced with 
virtual meetings by video call or telephone; they were scheduled using the same system as the 
in-person meetings and used the same text message reminders. (Box 2.2 describes in more detail 
the impacts of the pandemic on the students, the DPP program, and the study itself.) 


STUDENT SURVEY 


During the third relative semester following random assignment for each cohort, MDRC and 
YPL fielded a student survey to the full study sample. The goals of the survey were to understand 
students’ experiences in college, and for the program group students in particular, to understand 
their experiences in the Detroit Promise Path program. The survey asked questions about stu- 
dents’ first year in college as well as whether they had enrolled in a second year. 
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FIGURE 2.2 Coaching Session Attendance Rates 
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SOURCE: MDRC calculations using data from the Detroit Promise Path colleges and the MDRC management 
information system. 


NOTES: Only coaching sessions attended by program group students during semesters in which they were enrolled in 
college are shown above. 

Coaching sessions for students who enrolled at colleges other than the ones they identified on their Detroit Promise 
scholarship applications are not included. 

The sample for this figure is the 829 program group students. 

For the purposes of this table, enrollment is defined as being enrolled at the college of random assignment only. 


Fielding 


Students were sent both physical mailings and email invitations to participate in the survey. 
Nonrespondents received a total of five follow-up email reminders over the ten-week data col- 
lection period plus phone calls. For Cohort 1, a $2 pre-incentive was included in the physical 
mailing. Following low response rates for that cohort, students in Cohort 2 were offered a $20 
gift card incentive that was sent to participants after their responses were recorded. Overall, 
the response rates for program group students were 34 percent for Cohort 1 and 42 percent for 
Cohort 2. Only surveys with more than half of the questions answered were considered complete 
and included in the analysis. 


Control group students were also included in the survey fielding, but the response rates were very 
low—notably lower than for program group students. For this reason, MDRC and YPL did not 
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Box 2.2 


The Effects of COVID-19 on Detroit Promise Path 


The COVID-19 pandemic struck Detroit early. In March and April 2020, it was among the hardest-hit 
cities in the United States. The impacts were immediate and dire for Detroit’s community college 
students, the focus of the Detroit Promise Path (DPP) study. 


Like other Detroit schools, the five community colleges in this study closed quickly. Some were able 
to pivot immediately to online education, while others took several weeks to reorganize and restart 
the semester virtually. Either way, students in the study experienced great disruptions in their lives 
and in their education. Many found themselves without reliable home internet or access to a laptop or 
other devices to use for their schoolwork. High unemployment rates in Detroit created great financial 
uncertainty for students, their families, and the city itself. All were still navigating the pandemic and its 
effects—many students had friends or loved ones who became ill or died—at the time of the writing of 
this report. 


The pandemic shutdown in Detroit began during the spring 2020 semester, the sixth semester of 
the DPP study’s second cohort, and both students and study were affected. Twenty-seven percent 
of study students in this cohort were still enrolled in the spring 2020 semester. To examine the 
pandemic’s impact, the MDRC research team used a framework developed by Larry Hedges and 
Elizabeth Tipton for assessing educational studies during the pandemic.” 


The research team conducted qualitative phone interviews with program staff to understand how 
DPP was operating during the pandemic. The program quickly moved from an almost exclusively 
in-person format to an exclusively remote one. In-person coaching meetings were replaced with 
video chat or phone calls. DPP program staff mobilized to get laptops to students who didn’t have 
them and who were now forced to take all classes from home. The study’s service contrast, or the 
difference between what students in the program group received and what students in the control 
group received, appeared to continue to be strong throughout the spring semester; program group 
students’ participation rates remained high despite the move to the virtual format. To the researchers’ 
knowledge, the colleges did not institute any similar remote coaching program that control group 
students might have accessed. 


The research team also assessed changes in student needs. According to conversations with DPP 
staff, program group students in the spring semester experienced much higher levels of need than 
in past semesters. Because of the disruptions to their education, many students shared that they 
were struggling with motivation, attention span, and study skills, given the abrupt move to online 
coursework. A few students chose to delay graduation so they could graduate when their families 
could attend in person. Students also expressed concerns about their health or the health of their 
families, job loss, financial instability, and the risk of eviction, among other issues. Because Detroit 
was so hard-hit, and because the effects of the pandemic disproportionately affected low-income 
and Black residents—two groups that represent a large share of the participants in this study—it is 
clear that students in the spring 2020 semester faced unusually challenging obstacles. The program 
staff worked hard to help them, but the impacts of the pandemic are only beginning to be understood. 
Some issues, like the threat of eviction, were far beyond the DPP staff’s capacity to assist. 


The research team invites readers to consider this context when reviewing the outcomes for this 
program. 


“Larry Hedges and Elizabeth Tipton. 2020. “Addressing the Challenges to Educational Research Posed by 
Covid-19.” Working Paper 20-47. Evanston, IL: Northwestern Institute for Policy Research. 
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assess Statistically significant differences between the two groups’ responses, as comparing the 
two groups would produce results that are neither reliable nor generalizable. Rather, the survey 
findings were used to understand program group students’ experiences in the program. Their 
responses may provide insights to consider for future replications or expansions of the model. 


Survey Findings 


A central aim of the survey was to examine participation in the Detroit Promise Path program, 
utilization of campus services outside of DPP,‘ self-efficacy,® and barriers to enrollment. The 
survey also included questions about student employment. The summary of findings presented 
here combines responses from the two cohorts. 


Program Participation and Service Usage on Campus 


A key component of DPP is the coaching support. Program group students’ participation rates 
in this service were high: Of those who responded to the survey, about 90 percent reported 
meeting with a coach. Over half of the program group survey respondents reported meeting 
with their coach more than three times. In their meetings with coaches, respondents reported 
covering issues such as financial aid, advice on academic progress, career planning, and personal 
challenges. Overall, participants found Detroit Promise Path worthwhile, with 86 percent of 
respondents noting that the program had been valuable. 


A large segment of respondents (76 percent) indicated that the monthly gift card incentive they 
received had been helpful. Participants reported that they most often spent their monthly gift 
card on food (44 percent), followed by transportation (22 percent) and books/supplies (19 percent). 
The reported use of the gift cards indicates that program participants were facing challenges 
paying for basic needs while in school. 


Self-Efficacy and Barriers to Enrollment 


Students in the program group reported high rates of familiarity with various college processes: 
73 percent of respondents reported that they understood the process of choosing classes well, 
and 7o percent reported understanding academic requirements for their program. Sixty-six 
percent of respondents said that they were familiar with the financial aid application process. 


4. Literature on student persistence and development points to on-campus involvement as one important 
factor in success, including grades and persistence to the second year, particularly among students 
who do not live on campus, students who work, students of color, and students who are less prepared 
academically. See for example Deil-Amen (2011); Kuh et al. (2008); Pike, Kuh, and Massa-MckKinley (2008); 
Pretlow, Jackson, and Bryan (2020). 


5. Internal characteristics such as motivation and a sense of self-efficacy are correlated with student 
success. Self-efficacy, or the evaluation of one’s ability to succeed, has been identified as a strong 
predictor of GPA and credit accumulation. See Beachboard, Beachboard, Li, and Adkison (2011); Braxton 
(2000); Cooper (2014); Duckworth, Peterson, Matthews, and Kelly (2007); Zajacova, Lynch and Espenshade 
(2005). 
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When it comes to self-efficacy and access to resources, 87 percent of respondents said they had 
the skills necessary to succeed in college, and 85 percent said they had the necessary support. 
Similarly, 86 percent believed they had the access to the academic resources they needed. The 
ratings were a bit lower regarding financial resources, with just over half of respondents agreeing 
that they had access to the financial resources they needed. About 61 percent of participants 
reported that they were motivated to finish college. About 85 percent of respondents indicated 
that they had decided on a major and a career path. 


Transfer rates were higher in this project than in other MDRC studies, as reflected in both the 
student survey findings and in the administrative data collected as part of the RCT (see Chapter 
3). Approximately 13 percent of respondents indicated that they had transferred to a different 
college since their initial college enrollment. When describing reasons for their transfer, respon- 
dents primarily indicated that they transferred due to access to transportation and wanting to 
enroll in classes closer to home. Other reasons for transfer included financial considerations or 
wanting to enroll in a different program of study. 


Program group survey respondents who were not enrolled in college at the time the survey was 
fielded were encountering several barriers, principally those related to the ability to meet basic 
needs: 52 percent of participants ranked finding reliable transportation to school as one of their 
top three barriers to enrollment, and 40 percent faced challenges in paying for transportation. 
Approximately 29 percent of respondents listed paying rent as a barrier, and 22 percent said the 
need to spend more time at work was a barrier to their continued enrollment. Over 75 percent 
of participants indicated that they held a job, with more than half of all respondents working 
20 or more hours per week. Taken together, these issues reflect the fact that basic needs were a 
challenge for this population and affected their ability to continue their education. 


These findings were echoed in student focus groups. Students frequently identified financial issues 
as barriers to success; transportation issues were also a common refrain. Students also struggled 
with motivation, organization, and study skills. As one student said, “The biggest struggle I face 
is not letting things get me down or into a depressed state of mind.” Several students said that 
adapting to the college environment, with “faster paced classes” and “needing to manage my 
study time,” was also a challenge. Others talked about mindset being important: “I have to give 
it all I can,” said one. “Nothing can keep me from succeeding,” said another. 


The survey and focus group findings underline an important consideration for both the Detroit 
Promise Path program and for other college-access programs in Detroit. Students highly valued 
the assistance from DPP and their relationships with their coaches. Yet financial barriers re- 
mained significant: Many students reported that they did not have sufficient financial resources 
to complete college and said cost concerns were a large barrier to their enrollment. A large seg- 
ment of the respondents also reported working more than part time, which affected their ability 
to take courses and make progress toward a degree. Structural barriers, most notably the lack 
of reliable, organized regional transit, also had an impact on students’ continued enrollment. 
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Effects on Students’ Academic Outcomes 


This chapter presents findings on the estimated effects of the Detroit Promise Path (DPP) pro- 
gram on students’ academic progress and college completion rates during the three years after 
they entered the evaluation. This study estimates the value-added of the new services offered 
by DPP over and above the Detroit Promise scholarship and a college’s typical support services. 


ABOUT THE EVALUATION 


As discussed earlier, this evaluation includes two cohorts of students who had fully completed 
the Detroit Promise scholarship process between 2016 and 2017. Through the process of random 
assignment, 1,268 students were enrolled into the study; 829 were assigned to the program group 
and 439 were assigned to the control group. The study examines the effect of offering students 
the chance to participate in DPP, knowing that some of them would not take advantage of this 
offer. Stakeholders, for example, the Detroit Regional Chamber, cannot (nor do they wish to) 
force students to participate in DPP. Understanding the effect of the offer to participate repre- 
sents what stakeholders can reasonably expect to achieve with the program. 


To estimate this effect, the outcomes of all students who were offered DPP (the program group), 
whether or not they enrolled, are compared with the outcomes of all students who were offered 
the Detroit Promise scholarship only (the control group). That is, the analyses estimate the effect 
of the “intent-to-treat.” This context is important for interpreting the results because around 
one-quarter of the students who were offered DPP did not enroll in college in the first year, 
and received no or very limited DPP services, and therefore probably didn’t benefit from the 
program. Even so, these students are part of the study’s analysis sample. They received initial 
email outreach from the DPP coaches as well as some follow-up text messages; however, they did 
not receive the bulk of DPP program services. For instance, they did not see a coach or receive 
financial incentives, as they were not enrolled in school. 


In any intent-to-treat analysis, some program group members do not receive program services; 
that’s to be expected. In this study, however, the proportion of program group members who 
did not receive program services was higher than in most other such evaluations by MDRC. 
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There were two reasons for this. First, in most of MDRC’s postsecondary education evaluations, 
eligible students are actively recruited before agreeing to participate in the program and study. 
But this was an opt-out study, meaning all Detroit Promise scholarship applicants were randomly 
assigned.’ Consequently, in this study, participants did not have to express interest in the add-on 
services of Detroit Promise Path in order to join. A major benefit of this approach is that the study 
results are more generalizable, meaning that they apply to nearly all Detroit Promise scholarship 
applicants, not to a subset who expressed interest in the program. The approach also met two of 
the Detroit Regional Chamber’s goals: to make sure that all students were included in program 
outreach, rather than just those who sought out assistance, and to make the sign-up process as 
easy as possible. A potential drawback of the opt-out approach is that because students did not 
actively elect to be part of the study and the new program, some students were not interested 
and therefore were less likely to engage in the program and benefit from it.? 


The second reason a relatively high proportion of program group members did not interact with 
the program relates to the timing of study enrollment. In most of MDRC’s studies in postsecond- 
ary education, eligible students have already matriculated when they join an evaluation—that is, 
they have been admitted to college, have selected classes, and are recruited in person on campus. 
In contrast, random assignment in this study occurred behind the scenes, during the summer 
before school started; many students signed up for the program when they had registered for 
college but had not yet enrolled in classes. 


Because of these two elements of the study design, the sample includes a large number of stu- 
dents who signed up for the scholarship but never actually enrolled in college. These students’ 
outcomes are all reported in the analyses as zeroes, as they were not enrolled or accumulating 
credits. These zeroes draw the overall outcomes downward in both the program group and the 
control group. 


SUMMARY OF THE MAIN EFFECTS 


To assess the success of the program, three primary outcomes were prespecified in this study: 
enrollment, credits earned, and degrees earned. Table 3.1 presents these outcomes at the end of 
three years. 


Over the three years of the program, DPP helped students in the program group make more 
progress in higher education compared with students in the control group. The program had 
positive effects on the numbers of semesters enrolled and credits earned. However, it did not 
lead to additional credential completion at the three-year mark.* Thirty percent of program 


1. Students had the option to request not to be in the study during the application and informed consent 
process. However, no students chose to do this. 


2. This may accurately reflect real-life take-up rates, however. 


3. The main findings are similar even after adjusting the p-values for multiple hypothesis testing. Appendix 
Table B.1 presents p-values that have been adjusted using the approach described in Westfall, Young, and 
Wright (1993). 
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TABLE 3.1 Three-Year Academic Outcomes Summary 


Program Control 
Outcome Group Group Difference P-Value 


Average number of semesters enrolled 2.9 2.5 0.4*** 0.002 


Average number of semesters enrolled, categorical* 


0 18.2 20.1 -1.9 0.417 

1-2 32.6 40.5 -7.9"" 0.005 

3-4 19.2 17.5 1.8 0.436 

5-6 30.0 21.9 8.1*** 0.001 
Total credits earned 17.1 13.5 3.77** 0.001 
Earned a credential (%) 7.2 6.8 0.4 0.771 
Sample size (total = 1,268) 829 439 


SOURCES: MDRC calculations using data from the National Student Clearinghouse and the Detroit 
Promise Path colleges. 


NOTES: Estimates are adjusted by site, interaction between race and gender, and ACT and SAT score. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Weights are calculated to make the effective (weighted) random assignment ratio the same in all random 
assignment blocks. The effective random assignment ratio is equal to the full sample’s random assignment 
ratio. 

Credential measures include three full academic years of data (in other words, fall, spring, and summer 
for three full years). Enrollment and credit measures include these same data except the final summer 
session (in other words, Year 3 summer is not available). 

aAverage number of semesters is the primary outcome in this analysis. The categorical version of this 
outcome is a complementary measure added to aid the interpretation of the primary measure. 


group students enrolled in five or six semesters compared with 21.9 percent of control group 
members, for an estimated impact of 8.1 percentage points. Program group students also earned 
significantly more credits than the control group students, on average—17.1 credits compared 
with 13.5 credits, respectively, a 27 percent increase—for an estimated impact of 3.7 credits. At 
the end of three years, there was not a substantial effect on earning a degree or certificate: 7.2 
percent of the program group earned a degree or certificate compared with 6.8 percent of the 
control group. 


OVERALL PROGRAM EFFECTS ON ACADEMIC OUTCOMES 


Enrollment 


DPP helped more students enroll and persist in college. Figure 3.1 presents enrollment levels for 
each research group during the three-year follow-up period.‘ In the first semester, 65 percent 


4. Appendix Table B.3 provides additional information for the measures discussed in this section. 
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FIGURE 3.1 College Enrollment Rates by Semester 
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SOURCE: MDRC calculations using data from the National Student Clearinghouse and the Detroit Promise 
Path colleges. 


NOTES: Estimates are adjusted by site, cohort, ACT and/or SAT score, and the interaction of gender and 
race/ethnicity. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Weights are calculated to make the effective (weighted) random assignment ratio the same in all random 
assignment blocks. The effective random assignment ratio is equal to the full sample’s random assignment 
ratio. 

The total sample size is 1,268 (program group = 829, control group = 439). 


of the program group students enrolled in college. During that same semester, 61 percent of 
control group students enrolled in college. The 4.4 percentage point difference represents the 
estimated impact of DPP on first-semester enrollment (p-value = o. 115). In Semesters 2 and 3, 
the estimated impact grew to 7.6 and 8.3 percentage points, respectively, with a smaller estimated 
effect beginning in Semester 4. In Semesters 5 and 6, the estimated effect was 4.1 percentage 
points (p-value = o. 114). 


As is typically seen in postsecondary evaluations, enrollment dropped steadily over the follow- 
up period. There was a substantial drop between the second and third semesters of about 18 
percentage points for both research groups: Program group enrollment dropped from 63 percent 
to 45 percent, and control group enrollment dropped from 55.4 to 36.7 percent. For 97 percent of 
the study sample, this time period represents the transition period between the end of students’ 
first spring semester and the start of their second fall semester (in other words, the start of the 
second academic year). Such drop-offs during this time frame are quite common.®* Some of the 
reasons identified in the research include financial aid issues in a new FAFSA year, students 


5. Pretlow, Jackson, and Bryan (2020); Kuh, Cruce, Shoup, and Gonyea (2008). 
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leaving school to work, students losing motivation or a connection to the college during the sum- 
mer, and students experiencing adverse life events that prevented reenrollment.® In an attempt 
to stave off some of these issues, the DPP program model included a component that encour- 
aged students to continue to make progress toward their goals over the summer. The program 
successfully increased summer enrollment in this period (more on this below) and maintained 
the enrollment impact from Semester 2 to Semester 3. However, since both research groups saw 
a sizeable drop in this time frame, it suggests that there is more to be done to help students 
through this transition. By the end of the follow-up period, 29.7 percent of the program group 
and 25.6 percent of the control group were still enrolled in college. 


Looking at a distribution of the number of semesters enrolled provides additional insights. As 
shown in Table 3.1, 60.6 percent of the control group enrolled in 0, 1, or 2 semesters compared 
with 50.8 percent of the program group. In other words, the program induced an additional 
9.8 percent of the program group students to enroll in 3 or more semesters. In fact, the largest 
increase is seen in enrolling in 5 or 6 semesters (8.1 percentage points). This implies that the 
program helped more students stay engaged with college longer, a first step in helping students 
make academic progress. 


Summer Enrollment 


DPP nearly doubled the proportion of students enrolling in summer courses during the first 
program summer (that is, the summer between Semesters 2 and 3). As mentioned, coaches en- 
couraged students to continue to make progress toward their goals over the summer, either by 
enrolling in summer courses or by participating in a local summer jobs program called Grow 
Detroit’s Young Talent. Among the program group, 17.9 percent enrolled in courses compared 
with 9.8 percent of the control group, for an estimated impact of 8.1 percentage points. During 
this period, program group members also earned 0.7 credits, on average, compared with 0.3 
credits in the control group (see Appendix Table B.2). 


These impacts do not continue into the second program summer (that is, the summer between 
Semesters 4 and 5). Taking a closer look at these findings by calendar year (not shown), it is 
apparent that the control group began enrolling in summer courses at higher levels in summer 
2018. This is likely because of a change in federal policy to reinstate year-round Pell (also known 
as Summer Pell) in the 2017-18 academic year. Since much of DPP’s early summer impacts were 
due to program messaging that summer tuition was covered by the Detroit Promise scholarship, 
analogous messaging about year-round Pell covering tuition costs sent out by the colleges to all 
students likely contributed to the higher control group summer enrollment rates. 


Full-Time Enrollment 


DPP induced more students to enroll full-time throughout the first two years. Estimated impacts 
range from 4.6 to 11.3 percentage points. The Detroit Promise scholarship—which both research 


6. Feldman (2017); Bailey et al. (2016); Bailey, Jaggars, and Jenkins (2015). 
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groups were eligible to receive—had a nominal full-time enrollment requirement, giving both 
research groups an incentive to enroll full time. (As a reminder, students were told to enroll full 
time, but they did not lose their scholarship if they dropped below full-time status.) While the 
additional components in the DPP group were not tied to full-time enrollment, the combina- 
tion of coaching support and additional resources made full-time more salient and feasible for 
a portion of students (see Figure 3.2). In Semesters 5 and 6, the estimated effect begins to shrink 
(2.2 percentage points and 0.8 percentage points, respectively). 


FIGURE 3.2 College Full-Time Enrollment by Semester 
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SOURCE: MDRC calculations using data from the Detroit Promise Path colleges. 


NOTES: Estimates are adjusted by site, cohort, ACT and/or SAT score, and the interaction of 
gender and race/ethnicity. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Weights are calculated to make the effective (weighted) random assignment ratio the same in 
all random assignment blocks. The effective random assignment ratio is equal to the full sample’s 
random assignment ratio. 

The total sample size is 1,268 (program group = 829, control group = 439). 


The proportion of both groups enrolling full time dropped over time. While this is partly 
due to overall enrollment declining, it also appears that the proportion of enrolled students at- 
tending full time dropped from year to year. For example, more than half of enrolled program 
group members were enrolled full time in Semesters 1 and 2. Only about a quarter of enrolled 
program group students were full time in Semesters 5 and 6.’ 


7. Roughly 10 percent of students in any given semester were enrolling at a college other than the college 
they indicated at the time of random assignment. Due to data limitations, full-time enrollment data are 
only available for students who were enrolled at their original college of random assignment; unlike 
enrollment and graduation status, the National Student Clearinghouse (NSC) data do not contain full-time 
enrollment or credits. As a result, the outcome levels for full-time enrollment are underestimated. These 
underestimates affected the program group and control group roughly equally. 
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Credit Accumulation 


DPP increased cumulative credits earned throughout the six-semester follow-up. Figure 3.3 rep- 
resents cumulative total credits earned (both developmental and college-level) during Semesters 
1 through 6. The effect on cumulative credits earned grew each semester. At the end of three 
years, the program group was ahead by 3.7 credits, on average, representing a 27 percent increase 
over the control group average of 13.5 credits.® 


FIGURE 3.3 Cumulative Credits Earned by Semester 
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SOURCE: MDRC calculations using data from the Detroit Promise Path colleges. 


NOTES: Estimates are adjusted by site, cohort, ACT and/or SAT score, and the interaction of gender and race/ 
ethnicity. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Weights are calculated to make the effective (weighted) random assignment ratio the same in all random 
assignment blocks. The effective random assignment ratio is equal to the full sample’s random assignment ratio. 

Total sample size is 1,268 (program group = 829, control group = 439). 


This impact is meaningful, as credit accumulation is an important indicator of academic prog- 
ress. At the same time, the overall level of credits earned by both groups after three years was 
quite low. If all students were consistently enrolled in college for three years, one would expect 
the average credits earned to range from 36 credits (for part-time students) to over 60 credits 
(for full-time students). The relatively low levels of average credits earned in this study reflect 


8. As with full-time enrollment, data on credit accumulation are not available from NSC. As a result, the 
outcome levels for credits earned were underestimated. These underestimates affected the program and 
control group roughly equally. 
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the large number of students who never enrolled in college (roughly 19 percent) or dropped out 
after 1 or 2 semesters (about 35 percent). 


Graduation 


After three years, there was not a substantial effect on graduation. As shown in Table 3.2, after 
three years (or six semesters), 7.2 percent of the program group earned any degree or certificate 
compared with 6.8 percent of the control group. Most of the credentials earned were associate 
degrees, with the remainder largely comprised of certificates. It is worth noting that a small 
portion of both research groups attended four-year institutions during this time. The percent 
ranged from roughly 3 percent to 6 percent, with no discernable differences between the research 
groups (see Appendix Table B.3). 


TABLE 3.2 Credentials Earned 


Program Control 
Outcome Group Group Difference P-Value 


Earned a credential from any college 


Semester 1 0.1 0.0 0.1 0.382 
Semester 2 0.3 0.7 -0.4 0.426 
Semester 3 0.8 0.7 0.2 0.745 
Semester 4 2.6 2.4 0.2 0.860 
Semester 5 4.0 3.6 0.5 0.689 
Semester 6 7.2 6.8 0.4 0.771 


Highest credential earned 


Certificate 1.7 2.5 -0.8 0.340 

Associate’s degree 5.2 4.0 1.2 0.312 

Bachelor’s degree or higher 0.3 0.0 0.3 0.160 
Missing degree type 0.0 0.2 -0.2 0.160 
Sample size (total = 1,268) 829 439 


SOURCES: MDRC calculations using data from the National Student Clearinghouse and the 
Detroit Promise Path colleges. 


NOTES: Estimates are adjusted by site, interaction between race and gender, and ACT and SAT 
score. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Weights are calculated to make the effective (weighted) random assignment ratio the same in 
all random assignment blocks. The effective random assignment ratio is equal to the full sample’s 

random assignment ratio. 
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Given DPP’s positive effects on enrollment and credit accumulation, it is worth considering 
why there were no positive effects on credential completion throughout three years. It may be 
that the program was not comprehensive enough to help all students, especially given the high 
level of financial need students expressed during surveys and interviews. For example, as shown 
above, roughly half of students in the program group either did not enroll in college at all or 
dropped out after one or two semesters. For students who did stay enrolled, the high rates of 
part-time enrollment may mean that the three-year follow-up period of this study was too soon 
to see degree impacts. This might be especially pronounced for students who enrolled in col- 
lege requiring remedial or developmental courses, as these courses lengthen the time it takes 
for students to earn a degree. 


To provide some insight into how close students may be to earning a degree, Table 3.3 presents 
the status of students at the end of the follow-up period. The top panel shows student enrollment 
information for the roughly 7 percent of the study sample who earned a credential by the end of 
Semester 6. Most of these students were enrolled in a two-year college, suggesting that they had 
either just finished their credential or were continuing to work on additional credentials (for 
example, they may have earned a certificate and were now working toward an associate’s degree). 


The second panel shows the status of the roughly 93 percent of the sample who had not earned 
a credential in this time frame. A majority of both research groups were no longer enrolled in 
college during the final study semester (68.5 percent of the program group and 72.2 percent of 
the control group). However, 24.5 percent of the program group and 21.5 percent of the control 
group were still enrolled in college. Roughly 5 percent of both groups were enrolled in a four-year 
college and the remainder were enrolled in a two-year college. Looking at credit accumulation 
for students enrolled in a two-year college (where credit information was available) shows that 
6.9 percent of the program group had earned 48 or more credits compared with 5.0 percent of the 
control group, for a difference of 1.9 percentage points. These students were near the 60-credit 
associate's degree threshold and were still enrolled, so they provide a rough sense of who might 
earn a credential in the next year. MDRC is seeking additional funding to track students’ longer- 
term outcomes to see whether, as degree receipt increases in later years, there is an impact of 
being offered Detroit Promise Path. 


Subgroup Findings 


The above presented the overall average effect of DPP on students’ academic progress and comple- 
tion. As part of an exploratory analysis, Figures 3.4 and 3.5 present DPP’s effects on credits and 
credentials earned, respectively, for various subgroups of students. Subgroups include students 
living with a parent who had earned a bachelor’s degree (or not), students who completed the 
scholarship application before or after the deadline (as a measure of risk), study cohort, and 
gender. DPP appears to be effective at increasing credits earned for all of these groups. As was 
seen for the overall sample, there were no discernable effects on earning a credential for any of 
these subgroups. 


Also of interest, given the variation in program implementation, are the effects of the program 
on credits and credentials earned by college, as shown in Figures 3.4 and 3.5. Because the sample 
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sizes at College 2 through College 5 are so small, ranging from 105 students to 256 students, as 
shown in Appendix Tables B.4 and B.5, the estimates presented here are imprecise—notice the 
wide confidence intervals. However, the amount of variation in the college-specific effect esti- 
mates on both outcomes is substantial enough to consider that it may represent real differences 
in the effectiveness of the program across the colleges. With respect to credits earned, four col- 
leges had positive effect estimates, and one college had a negative effect estimate. The college 
with the negative effect estimate is the college that struggled with program implementation and 
had lower levels of program participation. This lower level of program quality and fidelity to 
the model may explain the differences in credits earned here. See Appendix Tables B.4 and B.5 
for more detailed information on outcomes by college. 


TABLE 3.3 Student Status at Three Years 


Program Control 


Outcome (%) Group Group’ Difference P-Value 
Earned a credential 7.2 6.8 0.4 0.771 
Enrolled in a 4-year college (during final study semester) 0.8 0.6 0.2 0.726 
Enrolled in a 2-year college (during final study semester) 4.6 4.0 0.6 0.598 
Not enrolled (during final study semester) 1.8 2.2 -0.4 0.655 
Has not earned a credential 92.8 93.2 -0.4 0.771 
Enrolled in a 4-year college (during final study semester) 5.3 4.5 0.9 0.503 
Enrolled in a 2-year college (during final study semester) 19.2 17.0 2.2 0.329 
Earned 48 credits or more 6.9 5.0 1.9 0.171 
Earned 36 - 47 credits 3.5 2.6 1.0 0.336 
Not enrolled (during final study semester) 68.5 72.2 -3.8 0.160 
Sample size (total = 1,268) 829 439 


SOURCES: MDRC calculations using data from the National Student Clearinghouse and the Detroit Promise 
Path colleges. 


NOTES: Estimates are adjusted by site, interaction between race and gender, and ACT and SAT score. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Weights are calculated to make the effective (weighted) random assignment ratio the same in all random 
assignment blocks. The effective random assignment ratio is equal to the full sample’s random assignment ratio. 
Among students without a credential at three years, the outcomes for the measures enrolled in a 4-year 
college, enrolled in a 2-year college, and not enrolled sum to more than the total share of students without a 
credential. This is because two students enrolled in both a 2-year and a 4-year college during their final study 
semester. 
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FIGURE 3.4 Cumulative Credits Earned After Three Years, by Subgroup 
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SOURCES: MDRC calculations using data from the National Student Clearinghouse and the Detroit Promise Path colleges. 
NOTES: Estimates are adjusted by site, cohort, ACT or SAT score, and the interaction of gender and race/ethnicity. 

Cl = confidence interval. 

The total sample size is 1,268 (program group = 829, control group = 439). 
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FIGURE 3.5 Credentials Earned After Three Years, by Subgroup 
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SOURCES: MDRC calculations using data from the National Student Clearinghouse and the Detroit Promise Path colleges. 
NOTES: Estimates are adjusted by site, cohort, ACT or SAT score, and the interaction of gender and race/ethnicity. 

Cl = confidence interval. 

The total sample size is 1,268 (program group = 829, control group = 439). 
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The Cost of the Program 


This section calculates the cost of the Detroit Promise Path (DPP) program. Cost analysis 
quantifies the resources used for a program’s implementation along with their associated prices 
to estimate the cost of implementing the program. This information can inform decisions by 
policymakers, administrators, and practitioners interested in replicating such a program to 
similar effect. For those interested in replicating this program model, these cost analyses can 
serve as a useful starting point. 


MDRC’s analysis begins with calculating direct costs, or those directly related to the operation 
of the program,' such as administration and staffing and providing student support. Table 4.1 
breaks down direct costs into three main categories: administration and staffing, coaching, and 
monthly student financial incentives. 


TABLE 4.1 Direct Cost of the Program per Program Group Member per Year 


Program Component Cost ($) Percentage of Total (%) 
Administration and staffing 


Program administrators 114 17.6 
Other costs 28 4.3 

Subtotal 142 21.9 
Coaching 353 54.5 
Financial incentives 153 23.6 


Total Direct Cost 648 100.0 
SOURCE: MDRC calculations based on program expenditure data from the Detroit Regional Chamber. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 
Program costs are based on total costs during the first three years of the program. The discount rate 
used for program costs is 3 percent. All costs are shown in constant 2020 dollars. 


1. All direct cost categories include related facilities and overhead costs. 


The total direct cost per program group student per year was $648, for a total of $1,944 per stu- 
dent over the three years of the program. (This total includes program group members who did 
not enroll.) Direct costs make up the bulk of the total cost of the program. 


About 22 percent of the direct cost of the program—$142 per program group student per year 
—comes from administration and staffing: the fully dedicated program manager who oversees 
the operation of the program, supervises coaches, and provides quality control. Other profes- 
sional and contractual services, equipment, and materials (in the table labeled “other costs”) 
are also included in this category. 


Coaching activities, including interactions with students, data management, meetings and training, 
and program development, make up 55 percent of the program’s direct costs—about $353 per pro- 
gram group member per year. The financial incentives of $50 per month per student, contingent 
on program participation, make up 24 percent of the direct costs, averaging $153 per program 
group member per year (given the total number of incentives, on average, each student earned). 


These per-student cost averages are lower than they would have been if all students had stayed 
enrolled for the duration of the program. If these same direct costs were measured per enrolled 
student per year, they would be about three times as high as the dollar amounts above. For ex- 
ample, when more students stay enrolled in school, more coaches need to be hired to maintain 
desired caseload levels. Appendix Table C.1 presents more information and provides additional 
cost calculations. 


TECHNICAL ASSISTANCE COSTS 


MDEC staff provided technical assistance to the Detroit Regional Chamber, particularly during 
the startup phase. MDRC staff spent a total of approximately 815 hours on technical assistance 
that was not associated with the research study evaluation. Using national salary and benefit 
averages for Education Administrators—Management and Technical Consulting from the Bureau 
of Labor Statistics, this cost was approximately $62,000, or $21 per student per year. This cost 
is not included in the total cost presented but should be kept in mind for colleges looking to 
replicate the program. If included, the direct cost of the program per student would increase 
by 2.3 percent per year on average in the first three years. However, these costs are startup costs 
that would not continue indefinitely: 70 percent of these costs occurred in the first year, with 
only 2 percent coming in the third year. 


BASE, INDIRECT, AND TOTAL COSTS 


Table 4.2 breaks down the base cost, indirect cost, and total cost of the program. Base costs 
refer to the “usual” college services provided to students who are not in the DPP program—for 


2. U.S Bureau of Labor Statistics (2020). 
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TABLE 4.2 Net Cost of Education per Sample Member per Year from the College 
Perspective 


Program Control Difference 


Feature ($) Group Group (Net) 
Direct cost: cost of primary program components? 648 0 648 
Base cost: cost of credits attempted in the absence of the program 3,027 3,027 0 
Indirect cost: cost of additional credits attempted due to the program 366 0 366 
Upper bound: marginal cost equal to average cost® 732 0 732 
Lower bound: marginal cost equal to zero° 0 0 0 
Indirect revenue: tuition and fees (to college) 174 0 174 
Upper bound: marginal cost equal to average cost° 4,233 3,027 1,206 
Lower bound: marginal cost equal to zero*® 3,501 3,027 474 


SOURCE: MDRC calculations based on expenditure, transcript, and scholarship data from the Detroit Regional 
Chamber, and financial and enrollment data from the Integrated Postsecondary Education Data System 
(IPEDS). In-district tuition and fees are from the Michigan House Fiscal Agency. 


NOTES: Tests of statistical significance were not performed. 

Rounding may cause slight discrepancies in sums and differences. 

Program costs exclude external research costs. 

Credits attempted include all college-level and developmental credits attempted. 

“Direct cost as calculated in Table 4.1 

>"Marginal cost equal to average cost" represents the case in which existing college resources cannot be 
leveraged to accommodate changes in credits attempted, therefore incurring additional costs to the college. 
The additional costs to the college, or the marginal cost of the additional credits attempted, is approximated as 
the average cost per credit attempted at the institution. 

°"Marginal cost equal to zero" refers to the ability of existing college resources to absorb the cost of additional 
credits attempted by the program group without incurring new costs to the college. 


example, the cost of instructors, buildings, college administration, and so on. The total base 
cost provides context for interpreting the program’s direct cost. It was approximately $3,027 per 
student per year and is calculated by multiplying the cost per credit, as reported by the federal 
Integrated Postsecondary Education Data System (IPEDS), times the average number of credits 
the control group attempted per year. This means that the direct cost of the DPP program rep- 
resents an increase in cost of about 30 percent over the status quo. 


Indirect costs are those that result from changes in student behavior—in this case, the additional 
cost resulting from the DPP program causing students to stay enrolled and take more courses. 
Indirect costs are estimated based on the average number of additional credits attempted by the 
program students compared with the control group students. This analysis uses three approaches: 


= A lower-bound estimate assumes that the indirect costs equal zero—that is, that the college 
incurs no additional costs when more students enroll and/or when students attempt addi- 
tional credits. 
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= An upper-bound estimate is based on the average cost per credit attempted (as reported by 
IPEDS), including the cost of instruction, academic services, student services, and scholarships. 
It is unlikely that every additional credit attempted by a student costs the college as much as 
the average credit attempted, and it is also unlikely that there is zero cost to the college for 
additional credits attempted (before tuition revenues are considered). 


™ An average of the previous two estimates—the midpoint between the upper and lower bounds— 
is therefore used as the primary estimate of indirect costs: $366 per program group student 
per year. 


It is worth noting that from the college perspective, indirect costs are offset by increased revenue 
in the form of tuition and state funding support associated with the additional enrollments and 
credits attempted. Part of the indirect cost is covered by additional Detroit Promise scholarships 
extended to students who enrolled because of the program when they would not have otherwise. 
This is calculated by comparing the scholarship funds expended for students in the control group 
with the funds expended for those in the program group. The Detroit Regional Chamber ended 
up paying additional scholarship costs of about $150 per program group student per year as a 
result of improvements in enrollment and credit attempts (about two-fifths of the total indirect 
cost). With tuition and financial aid revenue from this and other sources, the indirect cost to the 
college from additional credit attempts was offset by $174 per program group per year. However, 
from a societal perspective, the additional resources used when these students take additional 
courses are an indirect cost.* 


The total costs per group member are presented in the final lines of Table 4.2. The total cost 
from the college perspective is calculated by adding the direct cost, base cost, and indirect cost, 
and subtracting the indirect (tuition) revenue.’ The total college-perspective cost per program 
group member per year was $3,867, compared with $3,027 to educate the average control group 
member. The net cost is defined as the difference between the total program group cost and the 
total control group cost. The college-perspective net cost was $840 per program group member 
per year, representing a 28 percent increase. 


COST-EFFECTIVENESS 


Table 4.3 breaks down the cost-effectiveness of DPP from the college perspective. The program’s 
average net cost over three years was $2,520 per student. This resulted in students earning an 
extra 3.7 credits, on average, and a near zero effect on degree completion. 


3. There is another indirect cost to society via the additional hours the student spends on credits they took 
because of the program. These student-time costs are not calculated here but might include time spent 
studying outside of class, time spent commuting to class, and foregone wages. Societal benefits from the 
program are also not calculated. 


4. State performance funding formulas can also provide an estimate of additional revenue to the college 
generated by increasing course attempts. However, the Michigan formulas applied to this program’s 
impacts yield less than a dollar per student in additional revenue transferred to colleges. 
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TABLE 4.3 Cost-Effectiveness Values from the College Perspective 


Program Control Difference 
Outcome Group Group (Impact) 


Direct cost of primary components ($) 1,944 0 1,944 
Base cost of credits attempted in the absence of the program ($) 9,081 9,081 0 
Indirect cost of additional credits attempted due to the program ($) 1,098 0 1,098 
Indirect revenue from tuition/additional credits attempted due to 

the program: ($) 522 0 522 
Cost per group member over three years ($) 11,601 9,081 2,520 
Earned a degree (%) 7.2 6.8 0.4 
Cost per degree earned ($) 161,125 133,551 27,574 
Total credits earned 17.4 13.5 3.7 
Cost per credit earned ($) 678 673 6 
Incremental cost per additional credit earned ($) 681 
Sample size (total = 1,268) 829 439 


SOURCE: MDRC calculations from program-specific participation and budget data, transcript data, and 
financial and enrollment data from Integrated Postsecondary Education Data System (IPEDS). 


NOTES: Rounding may cause slight discrepancies in sums and differences. All dollar values have been 
rounded to the nearest whole dollar. 

Tests of statistical significance have only been performed on outcome measures not costs. All outcomes 
are cumulative over three years. For these measures, a two-tailed t-test was applied to differences between 
research groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

"This revenue represents transfers to the college from students and government via tuition, financial aid, 
and scholarships. It is not a societal cost offset or benefit, but is included here to represent the college 
perspective. 


A cost-effectiveness analysis expresses the cost of interventions as cost per unit of a desired 
outcome. The status quo and program cost per outcome can be compared with each other, and 
the incremental cost per additional outcome caused by the program can be compared to that of 
other programs. The latter contrast might be more useful because comparing programs with 
similar impacts gives more than one estimate of the cost of achieving those impacts. This cost- 
effectiveness analysis considers the cost per degree earned within three years and the cost per 
credit earned. These estimates spread costs across all students who were offered the program, 
including those who enrolled less than full time, dropped out, or graduated. 


As noted above, Table 4.3 shows the cost-effectiveness calculations for the program from the 
perspective of the college over three years. The average cost per degree for the program group 
and the control group is the cost per group member divided by the percentage of those who 
received degrees. As shown in the table, the cost per degree earned was $27,574 more for the 
program group. This means the program was less cost-effective for this outcome compared with 
the status quo. Table 4.3 also shows the average credits earned in three years for each group. The 
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average cost per credit earned for each group is calculated by dividing the cost per group member 
by the average credits earned, resulting in a $6 higher cost per credit earned for the program 
group than the control group. Finally, the incremental cost per additional unit of each outcome 
is presented. Since there was no significant impact on degrees, this number was not calculated. 
For credits earned, each of the additional 3.7 credits earned per student cost an additional $681 
from the college perspective. 


Unsurprisingly, these findings show that using the DPP program to increase graduation rates was 
not cost-effective at the three-year point because the estimated effect on degree completion was 
near zero. The DPP program was not cost-effective per credit earned compared with the status 
quo (which has lower average credits earned). However, if increasing credits earned is a goal, 
Detroit Promise Path may be a competitive option compared with other programs. Its relatively 
low cost point compared with other postsecondary interventions may make it appealing for those 
seeking to improve enrollment and credit accumulation rates for students. 
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Conclusions and Lessons for the Field 


The Detroit Promise Path builds evidence-based student support components into an existing 
College Promise scholarship program. Over the three years of this study, DPP helped more stu- 
dents make progress in college: Students in the program enrolled in more semesters and earned 
more credits, compared with those who were offered the scholarship alone. These are valuable 
improvements. At this point, however, there is no evidence of an increase in degrees earned. 
MDEC hopes to secure funding for longer-term follow-up of this study population in future 
years to continue tracking students’ graduation rates. 


Programs that produce large improvements in college graduation rates have been identified, 
but they are rare. Detroit Promise Path’s impacts on students’ overall enrollment patterns and 
credit accumulation are among the larger effects MDRC has found in its randomized controlled 
trials of such programs. Offering DPP is a strong step toward keeping students enrolled and 
earning credits, yet there is more to be done to help them graduate within three years. Nearly 
all students interviewed in the study expressed a highly positive view of DPP and of the help 
they received from coaches in particular. Yet these students continued to face great barriers 
to success, including financial uncertainty, inadequate academic preparation for college-level 
work, unreliable public transportation to school, and competing responsibilities at school and 
at home, among other challenges. Many of these issues will require greater systemic efforts with 
collaboration from stakeholders across policy domains to effect meaningful change. 


One significant issue that merits further research and experimentation is improving fall-to-fall 
persistence. In the DPP student population, as in many community colleges across the country, 
enrollment rates drop precipitously from one academic year to the next. While the DPP program 
had a positive impact on students’ likelihood of staying in school, the overall retention rates 
remained quite low. There is more to understand about the confluence of factors—from financial 
aid issues to academic under-preparedness to social disengagement from college life over the 
summer or “summer melt”—that might inform stronger interventions that could help students 
stay enrolled. While there has been some research into summer melt between high school and 
college, the continued issue in subsequent college years merits further investigation. In addi- 
tion, there may be policy changes that can help Detroit Promise students—and all students in 
Detroit—stay in school. 
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One element of DPP may explain the positive impacts on credit accumulation without a cor- 
responding impact on graduation rates at the three-year mark. While the Detroit Promise 
scholarship encourages students to enroll in college full time, students don’t lose their scholar- 
ship if they drop below full-time status. As shown in this report, students frequently enrolled 
part-time over the course of the study, making it very difficult to graduate within two or three 
years, even though they continued to accrue college credits. Instituting a full-time enrollment 
requirement has clear downsides. Many students have competing priorities such as work and 
child care. Serving only full-time students would certainly help improve the time it takes to earn 
a degree within the program, but doing so would cut out many of the highest-need students who 
stand to benefit the most from the program. For this reason, the Detroit Promise scholarship’s 
staff chose not to enforce a full-time requirement. Programs seeking to learn from this study 
may want to weigh the relative benefits and drawbacks of encouraging rather than enforcing 
full-time enrollment. 


LESSONS FOR THE FIELD 


College Promise programs and other college, placed-based scholarships—interventions aimed 
at improving college access by making tuition free—have grown in popularity in recent 
years. The Detroit Promise Path shows that these programs can also be leveraged to help 
students stay in school and earn more credits. What can policymakers, college practitioners, 
advocates, and College Promise program leaders learn from the Detroit Promise Path? 


™ It’s not just about access. Students also need help with college progress and college success. 
The Detroit Promise Path program model added evidence-based components to the Detroit 
Promise scholarship to help meet students’ needs once they enrolled. These components made 
a difference for students who might otherwise have enrolled but not persisted in college with 
the Promise scholarship alone. 


= Externally managed and staffed programs must build trust and buy-in on campus. Unlike 
many college advising or coaching programs, an off-campus third party organization runs 
and staffs DPP. At colleges in the study where leadership saw the program’s value, DPP oper- 
ated smoothly and students got a robust coaching relationship on top of their usual college 
experience. At one college, however, where the program was less popular with the leadership, 
the program did not run smoothly, and the coach was ultimately moved off campus and met 
with students in community centers or other public spaces. As a result, this college did not 
experience the same outcomes as the other schools did. 


= Students want, and need, individualized help and a trusting relationship. In DPP, coaches 
reach out to students “cold,” introducing themselves and encouraging students to set up in- 
person meetings. More than 90 percent of students in the study responded to this outreach, 
reflecting a tremendous appetite for support. Throughout the program’s three-year duration, 


students identified their relationship with their coach as the most valuable program component. 
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™ Students continue to face significant financial challenges and other issues that stop them from 
completing a degree. They need more support—financial, academic, and emotional—from 
all of the institutions in their lives. Help paying for textbooks, child care, and housing, as 
well as emergency aid for unexpected expenses like car repairs or medical bills, could have a 
meaningful impact on students’ ability to stay enrolled and graduate on time. 


™ Succinct and actionable messages help address students’ individual circumstances. Text 
messages and social media were the most effective outreach strategies—students respond to 
texts and social media posts on platforms they use daily. Making messages short, clear, and 
actionable led students to complete tasks—for instance, scheduling a coaching appointment 
or registering for courses. For more complex activities, such as navigating FAFSA verification, 
an in-person meeting is necessary. 


™ Continued relationships and active outreach can reengage students. Many students who in- 
tended to enroll in college but hit snags—for instance, with complicated financial aid applica- 
tions—were able to enroll in the following spring semester because of continued outreach from 
DPP coaches. Having someone in their corner who can connect them with other resources or 
help them identify time management strategies can help students stay focused and engaged. 


™ Students are most often derailed by issues outside of college. Despite the best efforts of coaches, 
many of the serious personal challenges students were experiencing were insurmountable. 
Students facing food insecurity, lack of transportation, eviction, homelessness, domestic vio- 
lence, and mental health crises simply cannot prioritize college attendance, for understandable 
reasons. For high-need student populations, much more than typical campus-based support 
is required. Stakeholders should take an expansive view of the systemic changes needed to 
dramatically improve community college graduation rates. 


The Detroit Promise Path program model is one way that College Promise programs can build 


on their college access goals to help students stay enrolled and earn more credits. Yet there’s 
more to be done to help these students get to graduation day within three years. 
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APPENDIX 


A 


Supplementary Survey Tables 


APPENDIX TABLE A.1 Detroit Promise Path Survey Summary, Program Group 


Respondents Only 


Responses 
College enrollment and aspirations 
Enrolled in the same college since fall 2016 (%): 
Yes 
No 
Decline to answer 
Somewhat and strongly agree with the statements below (%): 
| have decided on a major. 


| have decided on a career path. 


Readiness and self-efficacy 
Somewhat and strongly agree with the statements below (%): 
| have the skills necessary to succeed in college. 
| have the support necessary to succeed in college. 
| have the financial resources necessary to succeed in college. 
| have access to the academic resouces | need to succeed in college. 


| am motivated to finish college. 


Students indicated good or extremely good understanding of the following (%): 


Application process 
Applying for financial aid 
Academic requirements 


Choosing classes 


Barriers 
On average, how many hours do you work in a typical week? 
Not currently employed 
1-10 
11-20 
21-30 
31-40 
40 or more 
Among those who selected "No" to being enrolled in college: 
Why aren't you enrolled in college this semester? (%): 
Finding reliable transportation 
Paying for transportation 
Paying for rent 
Paying for food 
Paying for books, supplies, or lab fees 
Finding child care 


Sample 
Size 


306 
306 
306 


306 
306 


306 
306 
306 
306 
306 


306 
306 
306 
306 


306 
306 
306 
306 
306 
306 


88 
88 
88 
88 
88 
88 


Program 
Group 


83.5 
12.9 
3.1 


84.8 
86.7 


86.7 
84.7 
53.7 
85.6 
61.4 


66.0 
66.5 
70.0 
72.6 


23.7 
6.6 
17.0 
23.7 
19.3 
9.7 


52.0 
39.6 
29.1 
4.0 
13.5 
3.5 


(continued) 
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APPENDIX TABLE A.1 (continued) 


Sample Program 


Responses Size Group 
Paying for child care 88 0.0 
Needing to spend more time at work 88 22.4 
Finding time to study 88 2.3 
Having the skills to succeed academically 88 4.6 
Knowing where to go for academic support 88 6.4 


College services 


Since starting college, have you met with the following 3 or more times? (%) 


College coach 306 63.7 
Academic advisor 306 36.8 
Career advisor 306 18.5 
College financial aid advisor 306 34.6 
Other 306 9.0 


Have you discussed any of the following with faculty and staff other than your 
coach 3 or more times? (%) 


Help with class selection 298 72.7 
Academic progress 298 36.4 
Graduation requirements 298 41.8 
Choosing a major 298 43.9 
Career planning 298 40.9 
Applying for a job or internship 298 22.3 
Financial aid 298 61.6 
Other financial challenges 298 20.7 
Personal challenges 298 26.4 
Help with connecting to other student services 298 15.6 


Program feedback 
How do you typically spend the gift card you receive from the DPP? (%) 


Transportation 241 22.4 
Housing 241 5.7 
Food 241 43.6 
Books and other supplies 241 18.8 
Child care 241 0.6 
Other 241 5.4 
Percentage of respondents who found the following services helpful or 
very helpful: 
Meetings with coach 241 76.1 
DPP program as a whole 241 86.1 
DPP monthly gift card 241 76.4 


SOURCE: The Youth Policy Lab at the University of Michigan survey data and calculations. 
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APPENDIX TABLE A.2 Survey Response Bias in Program Students: 
Baseline Characteristics, Respondents vs. Nonrespondents 


Characteristic Respondents Nonrespondents Difference P-Value 
Gender (%) 
Female 63.0 55.5 742 0.035 
Male 37.0 44.5 -7.42 0.035 
Average age (years) 17.9 17.9 0.03 0.581 
Race/Ethnicity (%) 
Black or African-American 77.7 81.9 -4.17 0.154 
Hispanic or Latino 14.4 11.3 3.17 0.195 
Two or more races 3.9 4.6 -0.65 0.654 
Asian, Native Hawaiian, or other Pacific Islander 3.0 1.0 2.00 0.060 
White 1.0 1.3 -0.35 0.642 
American Indian or Alaska Native 0.0 0.0 0.00 
Does not live with a parent who has a bachelor's degree 75.8 84.1 -8.29 0.005 
Sample size (total = 829) 305 524 


SOURCES: MDRC calculations using survey data from the Youth Policy Lab at the University of Michigan and Detroit 
Promise scholarship application data. 


NOTE: One survey respondent is excluded from the table because they were not randomly assigned into the study. 
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APPENDIX TABLE A.3 Survey Response Bias in Program Students: 
Primary Outcomes, Respondents vs. Nonrespondents 


Outcome Respondents Nonrespondents' Difference P-Value 
Average number of semesters enrolled 3.9 2.2 1.65 0.000 
Average number of semesters enrolled, categorical 

0 6.9 24.8 

1-2 21.3 39.1 

3-4 23.9 16.6 

5-6 47.9 19.5 
Total credits earned 29.2 10.1 19.08 0.000 
Earned a credential (%) 15.1 2.5 12.60 0.000 
Sample size (total = 829) 305 524 


SOURCES: MDRC calculations using survey data from the Youth Policy Lab at the University of Michigan, the 


National Student Clearinghouse, and Detroit Promise Path colleges. 


NOTE: One survey respondent is excluded from the table above because they were not randomly assigned into the 


study. 
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Supplementary Academic Outcomes 
Tables 
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APPENDIX TABLE B.1 Primary Outcomes 


Program Group Control Group Estimated Effects 
Standard Standard Standard Adjusted? 
Outcome Mean Deviation Mean Deviation Difference Error P-value P-value 
Average number of semesters enrolled 2.85 2.13 2.48 2.08 0.38*** 0.12 0.0018 0.0060 
Total credits earned 17.14 21.80 13.47 19.96 3.67*** 1.15 0.0014 0.0060 
Earned a credential 7.20 25.74 6.76 25.38 0.44 1.50 0.7710 0.7510 
Sample size (total=1,268) 829 439 


SOURCE: MDRC calculations using data from National Student Clearinghouse and transcript data from the Detroit Promise Path colleges. 


NOTES: Estimates are adjusted by site, cohort, ACT and/or SAT score, and the interaction of gender and race/ethnicity. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Weights are calculated to make the effective (weighted) random assignment ratio the same in all random assignment blocks. The effective 
random assignment ratio is equal to the full sample’s random assignment ratio. 


APPENDIX TABLE B.2 Summer Enrollment 


Program Group Control Group Estimated Effects 
Standard Standard Standard 
Outcome Mean Deviation Mean Deviation Difference Error P-Value 
First program summer 
Enrolled 17.85 38.11 9.78 30.35 8.06*** 1.93 0.0000 
Credits attempted 0.90 2.37 0.43 1.94 0.47*** 012 0.0001 
Credits earned 0.68 2.03 0.32 1.55 0.37*** 0.10 0.0004 
Second program summer 
Enrolled 13.89 34.53 11.48 32.07 2.41 1.92 0.2104 
Credits attempted 0.63 2.15 0.54 2.10 0.09 0.13 0.4598 
Credits earned 0.50 1.88 0.39 1.75 0.11 0.11 0.3111 
Total (total = 1,268) 829 439 


SOURCES: MDRC calculations using data from the National Student Clearinghouse (NSC) and the Detroit Promise 
Path colleges. 


NOTES: Estimates are adjusted by site, interaction between race and gender, and ACT and SAT score. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Weights are calculated to make the effective (weighted) random assignment ratio the same in all random 
assignment blocks. The effective random assignment ratio is equal to the full sample’s random assignment ratio. 
NSC data are used for overall enrollment. Credits attempted and credits earned are based on data froma 
student's college of random assignment. 
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APPENDIX TABLE B.3 Enrollment, Credit, and Credential Impacts 


Program Group Control Group Estimated Effects 
Standard Standard Standard 
Outcome Mean Deviation Mean Deviation Difference Error P-Value 
Enrolled 
Semester 1 65.46 47.59 61.02 48.81 4.44 2.82 0.1154 
Semester 2 63.00 48.34 55.36 49.74 7.63*** 2.87 0.0080 
Semester 3 44.99 49.76 36.70 48.35 8.29*** 2.86 0.0038 
Semester 4 40.36 49.03 37.60 48.64 2.75 2.86 0.3351 
Semester 5 32.72 46.86 28.65 45.48 4.07 2.70 0.1322 
Semester 6 29.72 45.58 25.59 44.06 4.13 2.61 0.1142 
Enrolled full time 
Semester 1 37.13 48.33 31.05 46.36 6.09** 2.74 0.0265 
Semester 2 35.19 47.73 23.93 42.91 11.26*** 2.62 0.0000 
Semester 3 18.83 39.06 14.24 35.13 4,.59** 212 0.0303 
Semester 4 17.81 38.21 13.04 33.90 4.76** 2.08 0.0221 
Semester 5 8.42 27.71 6.21 24.32 2.21 1.52 0.1476 
Semester 6 6.51 24.69 5.69 23.17 0.81 1.37 0.5514 
Enrolled at a 4-year college 
Semester 1 3.92 19.25 2.83 17.00 1.09 1.09 0.3173 
Semester 2 2.96 16.71 3.97 19.95 -1.01 112 0.3696 
Semester 3 4.27 20.04 3.28 18.24 1.00 113 0.3786 
Semester 4 3.69 18.58 3.87 19.82 -0.19 114 0.8703 
Semester 5 5.31 22.27 5.59 23.31 -0.28 1.37 0.8394 
Semester 6 6.13 23.73 5.07 22.58 1.06 1.39 0.4466 


(continued) 
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APPENDIX TABLE B.3 (continued) 


Program Group Control Group Estimated Effects 
Standard Standard Standard 
Outcome Mean Deviation Mean Deviation Difference Error P-Value 
Credits earned 
Semester 1 4.54 5.20 3.95 4.93 0.58** 0.29 0.0416 
Semester 2 4.53 6.02 3.26 5.09 1.27*** 0.31 0.0001 
Semester 3 2.79 4.71 2.12 4.33 0.67*** 0.25 0.0082 
Semester 4 2.81 5.41 2.29 4.97 0.51* 0.29 0.0811 
Semester 5 1.73 3.72 1.24 3.28 0.49** 0.20 0.0135 
Semester 6 1.37 3.38 1.06 3.02 0.32* 0.18 0.0786 
Cumulative credits earned 
Semester 1 4.54 5.20 3.95 4.93 0.58** 0.29 0.0416 
Semester 2 8.94 9.90 713 8.98 1.81*** 0.53 0.0007 
Semester 3 11.58 13.28 9.16 12.08 2.42*** 0.70 0.0006 
Semester 4 14.26 17.36 11.32 15.95 2.94*** 0.92 0.0015 
Semester 5 15.90 19.82 12.53 18.10 3.37*** 1.05 0.0013 
Semester 6 17.14 21.80 13.47 19.96 3.67*** 115 0.0014 
Earned a credential 
Semester 1 0.10 3.24 0.01 0.00 0.09 0.10 0.3816 
Semester 2 0.35 5.88 0.70 8.38 -0.36 0.45 0.4265 
Semester 3 0.84 9.02 0.67 8.38 0.17 0.52 0.7448 
Semester 4 2.60 15.79 2.44 15.74 0.17 0.94 0.8601 
Semester 5 4.04 19.64 3.59 18.77 0.46 114 0.6890 
Semester 6 7.20 25.74 6.76 25.38 0.44 1.50 0.7710 
Sample size (total = 1,268) 829 439 


SOURCES: MDRC calculations using data from National Students Clearinghouse (NSC) and from the Detroit 
Promise Path colleges. 


NOTES: Estimates are adjusted by site, interaction between race and gender, and ACT and SAT score. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Weights are calculated to make the effective (weighted) random assignment ratio the same in all random 
assignment blocks. The effective random assignment ratio is equal to the full sample’s random assignment ratio. 
NSC data are used for overall enrollment. Credits attempted and credits earned are based on data from a 
student's college of random assignment. 
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APPENDIX TABLE B.4 Cumulative Credits Earned after Three Years, by Subgroup 


Credits Earned (Average) 


P-Value for 
Sample Program Standard __ P-Value for Differential 
Student Characteristic Size Group Control Group Difference Error Difference Estimated Effects 
Study college 0.042 tt 
College 1 573 16.90 13.04 3.86 1.62 0.018 
College 2 149 12.85 10.75 2.11 3.08 0.495 
College 3 185 18.39 8.57 9.81 ** 2.88 0.001 
College 4 105 12.85 7.59 5.26 = 2.84 0.067 
College 5 256 20.22 23.71 -3.50 3.27 0.286 
Sample size 1,268 
Living with a parent who has 
earned a bachelor's degree 0.640 
No 987 15.65 11.79 3.86 *** 1.23 0.002 
Yes 247 24.71 19.27 5.44 # 3.16 0.086 
Sample size 1,234 
Timeliness of scholarship 
application completion 0.935 
Before the deadline 829 17.51 14.04 3.46 * 1.45 0.017 
After the deadline 438 16.32 12.66 3.66 * 1.93 0.059 
Sample size 1,267 
Study cohort 0.832 
Fall 2016 589 18.35 14.78 3.57 ™ 1.72 0.038 
Fall 2017 644 16.02 12.95 3.08 ‘J 1.58 0.052 
Sample size 1,233 


(continued) 
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APPENDIX TABLE B.4 (continued) 


Credits Earned (Average) 


P-Value for 
Sample Program Standard __ P-Value for Differential 
Student Characteristic Size Group Control Group Difference Error Difference Estimated Effects 
Gender 0.4000 
Female 748 17.73 13.16 4.57 *** 1.54 0.0030 
Male 520 16.43 13.84 2.58 1.79 0.1490 
Sample size 1,268 


SOURCE: MDPC calculations using data from the Detroit Promise Path colleges. 


NOTES: Estimates are adjusted by site, cohort, ACT or SAT score, and the interaction of gender and race/ethnicity. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
A two-tailed t-test was applied to differences of effect between or among subgroups. Statistical significance levels are indicated as: ttt = 1 
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APPENDIX TABLE B.5 Credentials Earned After Three Years, by Subgroup 


Credentials Earned (%) 


Sample Program Control Standard P-Valuefor P-Value for Differential 
Student Characteristic Size Group Group Difference Error Difference Estimated Effects 
Study college 0.073t 
College 1 573 7.72 6.87 0.85 2.22 0.702 
College 2 149 4.84 0.57 4.27* 2.42 0.079 
College 3 185 4.86 0.00 4.86** 2.06 0.019 
College 4 105 2.64 8.67 -6.04 4.98 0.228 
College 5 256 10.42 16.57 -6.15 4.77 0.199 
Sample size 1,268 
Living with a parent who has earned 
a bachelor's degree 0.247 
No 987 6.86 5.22 1.64 1.60 0.305 
Yes 247 9.14 12.99 -3.85 4.46 0.389 
Sample size 1,234 
Timeliness of scholarship application 
completion 0.410 
Before the deadline 829 7.83 6.57 1.27 1.85 0.495 
After the deadline 438 5.91 7.34 -1.43 2.70 0.596 
Sample size 1,267 
Study cohort 0.828 
Fall 2016 589 6.47 5.90 0.57 2.07 0.783 
Fall 2017 644 7.83 7.92 -0.09 2.23 0.967 
Sample size 1,233 


(continued) 
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APPENDIX TABLE B.5 (continued) 


Sample 

Student Characteristic Size 
Gender 

Female 748 

Male 520 

Sample size 1,268 


Average Credentials Earned 


Program Control 
Group Group Difference 


8.12 6.28 1.84 
5.87 7.65 -1.78 


Standard 
Error 


2.01 
2.32 


SOURCE: MDRC calculations using data from the NSC and the Detroit Promise Path colleges. 


P-Value for 
Difference 


0.3620 
0.4440 


NOTES: Estimates are adjusted by site, cohort, ACT or SAT score, and the interaction of gender and race/ethnicity. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
A two-tailed t-test was applied to differences of effect between or among subgroups. Statistical significance levels are indicated as: ttt = 1 


P-Value for Differential 
Estimated Effects 


0.2390 


APPENDIX 


C 


Supplementary Cost Tables 


APPENDIX TABLE C.1 Direct Cost of the Program per Program Group 


Enrollee per Year 


Program Component Cost ($) 


Administration and staffing 


Percentage of Total (%) 


Program Administrators 355 17.6 
Other Costs 87 4.3 
Subtotal 442 21.9 
Coaching 1,102 54.5 
Monthly Incentives 478 23.6 
Total Direct Cost 2,022 100.0 


SOURCE: MDRC calculations based on program expenditure data from the Chamber. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 
Program costs are based on total costs during the first four years of the program. 


The discount rate used for program costs is 3 percent. All costs are shown in constant 2020 dollars. 
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